A GraphQL configuration file database

docker run -it \
    -p 9090:9090 \
    -v /tmp/data/mollydb:/var/mollydb/data \
    wesovilabs/mollydb:0.0.1-alpha

Project Status

mollydb is not stable at all, the API is in continuous definition and some architecture topics are not closed.

Learning from provided feedback to toward the project correctly.

About

MollyDB is a configuration file database that provides a GraphQL API.

It permits interacting with configuration files like If they were tables in a relational database system or documents in a document based database.

Architecture overview

MollyDB Hierarchy

MollyDB works in a distributed way and it's able to deal with storage that are hosted on different paths.

Hierarchy in mollydb is defined as:

Storage

A storage can be defined as a place where documents are hosted. So far, only local directories are supported but in the near future other places such as ssh directory, database collections or even git repository could be understand as a storage

Sample of storage would be:

  • file:/var/data/databases
  • file:/var/data/micro-services

Documents

As is said in the storage paragraph description, a storage contains documents. So far, only files with yml extension are supported.

For example, the storage database, in path file:/var/data/databases
contains the files: mongodb.yml, mysql.yml and ListOfDocuments.docs, but only the following will be understood as document.

  • mongodb.yml
  • mysql.yml

Property

A property is defined as any configurable value in a document For example, a document api-gateway.yml whose content is:

self:
  port: 7000
  address: 0.0.0.0
log:
  level: DEBUG
services:
  accounts:
    address: localhost:7001

provides the following properties:

  • self.port
  • self.addres
  • log.level
  • services.accounts.address

Scenario

A clearer scenario is showed below

mollydb

Registering a storage

Once mollydb is up and running we must proceed to register a storage in the system. We can do it by making use of the provided GraphQL API.

curl -XPOST http://localhost:9090/graphql \
  -H 'Content-Type: application/graphql' \
  -d 'mutation registerI18nDirectory { register
  (path:"resources/data/databases",  
        name:"db"){ name len documents { name } } }'

Once the storage is registered in mollydb, a daemon will be launch and it will monitor documents in the storage. In case of a new documentis created mollydb will be updated. The same occurs when a file is deleted or a file is modified.

Creating a hook

Mollydb provide you a way yo avoid being making connections every now and then if you need to be informed whena property is changed.

To do this we can make use of the hooks. So far, only Rest hooks are supported. Yo will create a hook by setting the property path you want to listen and the url and verb that will be invoked when the property changed.

RestHook

curl -XPOST http://localhost:9090/graphql \
  -H 'Content-Type: application/graphql' \
  -d 'mutation { addRestHook (uri: 
  "http://localhost:3000/properties/mongodb-port", verb: "PUT", path: 
  "mollydb://db/mongodb?key=database.port") }' 

What does the above code means?

When the poperty database.port in document mongodb (that belongs to the registered storage db) changes, then mollydb will make the next request PUT http://localhost:3000/properties/mongodb-port with the below body

{
    "path": "mollydb://db/mongodb?key=database.port",
    "key": "database.port",
    "value": 27030 
}

Running mollyDB

Download executables

From the code

  • Clone the repository into your local machine:
    git clone https://github.com/wesovilabs/mollydb.git
  • Checkout the chosen tag.
    git checkout <mollydb.tag>
  • Run the following command:
    make run 

Docker

Docker images are published as far as a new tag on master branch are created.

By default mollydb is launched on port 9090, so in case of you want to forwarding to a different one you will need to indicate it when running docker command

In order to being able to make use of local directories as storage in the container you will need to mount a volume when running docker.

Scenario: For the below sample we assume you have the path */tmp/data/mollydb on your local machine and no processes running on port 9090

     docker run -it -p 9090:9090 \
     -v /tmp/data/mollydb:/var/mollydb/data wesovilabs/mollydb:0.0.1-alpha

GraphQL

MollyDB provides a GraphQL API that can be consumed easily

Types

Types managed by GraphQL API are the below

Storage

A mollyDB storage.

Fields

  • documents: The list of documents that belong to this storage
  • len: The number of documents in the storage
  • name: The name of a storage

Document

A mollyDB document.

Fields

  • name: The name of the document.
  • len: The number of properties in the document.
  • properties: List of properties of the document

Property

Document content

Fields:

  • key: The key of the property
  • path: The path of the property
  • value: The value of the property

Queries

storageList(name: String = "any"): [Storage]

This query allow us to deep from the root of the mollyDB system until a Property definition.

Sample

request

query StorageQuery {
  storageList { # optional filter name
    name
    len
	documents(name: "mongodb") { # optional filter name
      name
      len
      properties { # optional filter key
        key
        value
        path
      }
    }
  }
}

response

{
  "data": {
    "storageList": [
      {
        "documents": [],
        "len": 6,
        "name": "i18n"
      },
      {
        "documents": [
          {
            "len": 4,
            "name": "mongodb",
            "properties": [
              {
                "key": "database.hostname",
                "path": "mollydb://db/mongodb?key=database.hostname",
                "value": "mongodb.wesovilabs.com"
              },
              {
                "key": "database.port",
                "path": "mollydb://db/mongodb?key=database.port",
                "value": "27030"
              },
              {
                "key": "database.credentials.username",
                "path": "mollydb://db/mongodb?key=database.credentials.username",
                "value": "root"
              },
              {
                "key": "database.credentials.password",
                "path": "mollydb://db/mongodb?key=database.credentials.password",
                "value": "secret"
              }
            ]
          }
        ],
        "len": 3,
        "name": "db"
      },
      {
        "documents": [],
        "len": 3,
        "name": "ms"
      }
    ]
  }
}

properties(storage:String = "any" document:String = "any" property:String = "any"): [Property]

Find properties by filtering records by the name of the storage or/and the name of the document or/and the key of the property. Default value for filters is any. The output is an array of type Property

Sample

request

query FindProperties {
  properties(storage: "ms", property: "log.level") {
    path
    key
    value
  }
}

response

{
  "data": {
    "properties": [
      {
        "key": "log.level",
        "path": "mollydb://ms/api-gateway?key=log.level",
        "value": "${mollydb://ms/global?key=log.level}:DEBUG"
      },
      {
        "key": "log.level",
        "path": "mollydb://ms/global?key=log.level",
        "value": "DEBUG"
      },
      {
        "key": "log.level",
        "path": "mollydb://ms/ms-account?key=log.level",
        "value": "DEBUG"
      }
    ]
  }
}

property(path: String): Property Find a property in any document of any storage by the connection path. This is an unique value for each property in all the mollydb system. The output is a single Property

Sample

request

query PropertyPath {
  property(path: "mollydb://db/mongodb?key=database.hostname") {
    path
    key
    value
  }
}

response

{
  "data": {
    "property": {
      "key": "database.hostname",
      "path": "mollydb://db/mongodb?key=database.hostname",
      "value": "mongodb.wesovilabs.com"
    }
  }
}

Mutations

register(name: String!path: String!): Storage

The purpose of this mutation is registering a new storage into mollyDB

Sample

request

mutation registerI18nDirectory {
  register(path: "resources/data/i18n", name: "i18n") {
    name
  }
}

response

{
  "data": {
    "register": {
      "name": "i18n"
    }
  }
}

unRegister(name: String!): String

The purpose of this mutation is unregister an existing storage from mollyDB

Sample

request

mutation unRegisterI18nDirectory {
  unRegister(name: "i18n")
}

response

{
  "data": {
    "unRegister": "storage deleted successfully!"
  }
}

propertyRestHook(uri: String! verb: String! path: String!): String

The purpose of this mutation is hook property and be notified when these have changed

Sample

request

mutation AddHook {
  propertyRestHook(
    uri: "http://localhost:3000/properties/mongodb-port", 
    verb: "PUT", 
    path: "mollydb://db/mongodb?key=database.port"
  )
}

response

{
  "data": {
    "propertyRestHook": "property hooked"
  }
}

Playing with GraphQL

GraphiQL is interated with MollyDB

Once the system si launched you can deal with MollyDB by making use of GraphiQL. GraphiQL is deployed in the same port that mollydb. So assuming you use the default port 9090, once you have launched molly you can open the browser and play with GraphiQL

graphiql

Below some examples of graphql queries and mutations used to test manually the system:

mutation registerI18nDirectory {
  register(path: "resources/data/i18n", name: "i18n") {
    name
  }
}

mutation unRegisterI18nDirectory {
  unRegister(name: "i18n")
}

mutation registerDatabaseDirectory {
  register(path: "resources/data/databases", name: "db") {
    name
  }
}

mutation registerMicroservicesDirectory {
  register(path: "resources/data/microservices", name: "ms") {
    name
  }
}

query StorageQuery {
  storagesList {
    name
    len
		documents(name:"mongodb"){ 
      name
      len
      properties {
        key
        value
        path
      }
    }
  }
}

query FindProperties {
  properties(storage: "ms", property: "log.level") {
    path
    key
    value
  }
}

query PropertyPath {
  property(path: "mollydb://db/mongodb?key=database.hostname") {
    path
    key
    value
  }
}

mutation HookQuery {
  propertyRestHook(uri: "http://localhost:3000/properties/mongodb-port", verb: "PUT", path: "mollydb://db/mongodb?key=database.port")
}

Links

MollyDB documentation can be found on Wiki:

Other Links

Architecture overview

Running mollydb

GraphQL

Playing with graphql

Changelog

The road map

Contributing

Owner
Wesovi Labs
Love what you do, enjoy doing it!
Wesovi Labs
Similar Resources

Go configuration made easy!

gofigure Go configuration made easy! Just define a struct and call Gofigure Supports strings, ints/uints/floats, slices and nested structs Supports en

Sep 26, 2022

Harvest configuration, watch and notify subscriber

Harvester Harvester is a configuration library which helps setting up and monitoring configuration values in order to dynamically reconfigure your app

Dec 26, 2022

go implementation of lightbend's HOCON configuration library https://github.com/lightbend/config

HOCON (Human-Optimized Config Object Notation) Configuration library for working with the Lightbend's HOCON format. HOCON is a human-friendly JSON sup

Dec 3, 2022

🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022

Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022

A golang package for parsing ini-style configuration files

Mini Mini is a simple ini configuration file parser. The ini syntax supported includes: The standard name=value Comments on new lines starting with #

Jan 7, 2023

A dead simple configuration manager for Go applications

Store Store is a dead simple configuration manager for Go applications. I didn't like existing configuration management solutions like globalconf, tac

Dec 24, 2022

Go configuration with fangs

Go configuration with fangs

Viper v2 feedback Viper is heading towards v2 and we would love to hear what you would like to see in it. Share your thoughts here: https://forms.gle/

Jan 8, 2023

shops is a simple command-line tool written in Go that helps you simplify the way you manage configuration across a set of machines.

shops is a simple command-line tool written in Go that helps you simplify the way you manage configuration across a set of machines. shops is your configuration management tool of choice when Chef, Puppet, Ansible are all too complicated and all you really want to do is run a bunch of regular shell against a set of hosts.

Jul 5, 2021
Comments
  • Hooks deletion

    Hooks deletion

    A way to delete an existing hook should be implemented. Currently we can create hooks (restHooks so far) by making use of mutation:

    mutation HookQuery {
      propertyRestHook(uri: "http://localhost:3000/properties/mongodb-port", verb: "PUT", path: "mollydb://db/mongodb?key=database.port")
    }
    

    So another mutation must be implemented in order to delete the hooks.

  • create hooks to more than a single property at time

    create hooks to more than a single property at time

    So far, we can only create hooks to single properties. It would be handy if we could do it for more than a property, for example for a . whole document or even properties that match with a pattern. Below some ideas to be considered

    mutation {
      propertyRestHook(
        uri: "http://wesovilabs.com/_config/database", 
        path: "mollydb://db/mongodb?key=database.(\w+)"
      )
    }
    
    mutation {
      propertyRestHook(
        uri: "http://wesovilabs.com/_config/database", 
        path: "mollydb://db/mongodb?key=(\w+)"
      )
    }
    
  • Support property value references

    Support property value references

    A property could reference to other one. I.e.

    storage name: i18n

    • default.yml greetings:"hi"
    • es.yml greetings:"hola"
    • en.yml greetings: ${i18n.default.greetings}
It syncronizes the configuration described in a YAML file against your GitHub Organization

It syncronizes the configuration described in a YAML file against your GitHub Organization. Combined with a CI system, it can be used to implement GitOps for GitHub.

Jul 19, 2021
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Jan 8, 2023
12 factor configuration as a typesafe struct in as little as two function calls

Config Manage your application config as a typesafe struct in as little as two function calls. type MyConfig struct { DatabaseUrl string `config:"DAT

Dec 13, 2022
JSON or YAML configuration wrapper with convenient access methods.

Config Package config provides convenient access methods to configuration stored as JSON or YAML. This is a fork of the original version. This version

Dec 16, 2022
Configure is a Go package that gives you easy configuration of your project through redundancy

Configure Configure is a Go package that gives you easy configuration of your project through redundancy. It has an API inspired by negroni and the fl

Sep 26, 2022
An opinionated configuration loading framework for Containerized and Cloud-Native applications.
An opinionated configuration loading framework for Containerized and Cloud-Native applications.

Opinionated configuration loading framework for Containerized and 12-Factor compliant applications. Read configurations from Environment Variables, an

Dec 16, 2022
Load configuration in cascade from multiple backends into a struct
Load configuration in cascade from multiple backends into a struct

Confita is a library that loads configuration from multiple backends and stores it in a struct. Supported backends Environment variables JSON files Ya

Jan 1, 2023
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Nov 3, 2022
A minimalist Go configuration library
A minimalist Go configuration library

fig fig is a tiny library for loading an application's config file and its environment into a Go struct. Individual fields can have default values def

Dec 23, 2022
go-up! A simple configuration library with recursive placeholders resolution and no magic.

go-up! A simple configuration library with placeholders resolution and no magic. go-up provides a simple way to configure an application from multiple

Nov 23, 2022