A distributed Configuration Center server that manages config in a container. The container is composed of fields (abstract layer includes: KV, LIST, DICT type). The Field contains basic datatypes (int, float, bool, string, list, dict).

cassem

Go Report Card go.de │ v reference

config assembler from key-value pairs' container which include basic datatypes, such as int, string, float, bool, list, dict

Features

  • HTTP Restful API.
  • Export container (config container) into different file format (JSON / TOML).
  • Manage CTL / UI support.
  • Master / Slave architecture support based raft (only write on master).
    • Master / Slave architecture and only allow to write on master.
    • Slave need to forward request to master.
  • RESTful API permission control.
  • Changes watching and notifying.
    • Watcher component.
    • Watcher Server.
    • Client SDK support.
  • Distributed Cache middleware to speed up the API which downloads container in specified format.

Documentation

- cassemctl

- cassemd

Benchmark

benchmark tested core RESTful API and try to optimize them, each benchmark test displays the final QPS result.

README

References

Owner
Comments
  • develop `cassemctl` command

    develop `cassemctl` command

    features:

    1. genconf generates the default config file for users.
    2. migrate command to initialize persistence component (for now, DB support).
    3. basic resource managing commands (CURD) + watch. Maybe, this would build on the client SDK ...
    4. health diagnosis (TODO). metrics and cluster infos.
  • move `cache` instance from `core.Core`

    move `cache` instance from `core.Core`

    For now, cache is held by the core.Core, but the cache component is mainly used in the raft.FSM. To close up all of the cache component entries, so move it into FSMWrapper maybe make it cleraly to read.

  • optimize join action

    optimize join action

    Background

    cassemd must get leader address so that it could join to cluster successfully, this is not easy to use and is unreasonable.

    Solution

    • forward join request to the leader while a slave node receives a join request
    • allow cassemd to receive an array of node address so that it could try to join one by one.
  • Remove pair concepts from cassem

    Remove pair concepts from cassem

    • Redesign the concepts of config container.
    • Focus on the edge problems in the situation of config center.
    • The reuse of pair maybe not the main issue on which the config center should take care.
  • feature/refactor persistence

    feature/refactor persistence

    • feat: adjust Repository constructor API
    • feat: independent convertor from repository
    • feat: reduce authorizer from server but in coordinator
    • refactor: pull away raft from core which makes core as daemon rather than a component.
    • refactor: merge user repo, repo and policy adapter.
  • persistence refactor

    persistence refactor

    Background

    Need a distributed persistence component to support what helps cassem become a system satisfied CP and provide available as much as possible. On another way, DO NOT want to import another dependency to make the save procedure complicated.

    Risk

    What is a risk is that would changes the cassem software architecture, even makes HTTP API changes.

    Thoughts

    bbolt would be used to replace mysql.

  • replace cache implementation with LRU-K(2) replacement algorithm

    replace cache implementation with LRU-K(2) replacement algorithm

    now cache components are implemented by go built-in map, it's too simple to manage caches. So that cassemd needs a new version of the cache component to manage caches.

    The duty of the cache component includes:

    // IStore required ICache to implement persist and restore functions so that the cache could be
    // save into file and restore from that.
    type IStore interface {
    	// Persist serializes cache into []byte data.
    	Persist() ([]byte, error)
    
    	// Restore apply data to override all the cache data, data is coming from Persist.
    	Restore(data []byte) error
    }
    
    // ICache represents the proxy to operate cache data,
    // also need to replace data while cache size is over it's limits.
    type ICache interface {
    	IStore
    
    	// Set returns wasSet means this key needs to synchronous, err means set failed.
    	Set(key string, v []byte) SetResult
    
    	// Get
    	Get(key string) ([]byte, error)
    
    	// Del if any error is returned, core.Core would not trigger synchronous of the cache.
    	Del(key string) SetResult
    }
    
    // SetResult represents what operations would be caused by ICache.Set, operations include:
    //
    // 1. NeedSync tells users that them should trigger setting apply.
    // 2. NeedDeleteKey should be set while Cache-Replacing happened or core need to delete cache by itself.
    //
    type SetResult struct {
    	err           error
    	NeedSync      bool
    	NeedDeleteKey string
    }
    
  • REFACTOR: cassemdb bootstrap procedure refactoring

    REFACTOR: cassemdb bootstrap procedure refactoring

    Now, cassemdb bootstrapped with fixed configuration of cassemdb cluster and it's members. But it should be another way:

    1. root node1 bootstrap or recover the cluster.
    2. node-2 to node-n join to the cluster which contains node1 only.
A course to build the SQL layer of a distributed database.

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

Jan 8, 2023
A simple golang api generator that stores struct fields in key/value based databases

Backgen A simple golang API generator that uses key/value based databases. It does not provide the database itself, only uses a interface to access se

Feb 4, 2022
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order

Jan 9, 2023
🛡️ Synchronize AdGuard Home config to replicas

AdGuardHome sync Synchronize AdGuardHome config to a replica instance. Current sync features General Settings Filters Rewrites Services Clients Instal

Dec 31, 2022
Parsing of numeric ranges from string

expand-range Parsing of numeric ranges from string. Convert "1,3-5" into [1,3,4,5]. Installation go get -u github.com/n0madic/expand-range Usage Impor

Nov 28, 2021
A rudimentary implementation of a basic document (NoSQL) database in Go
A rudimentary implementation of a basic document (NoSQL) database in Go

tiedot Documentation Keywords: Golang, go, document database, NoSQL, JSON tiedot - Your NoSQL database powered by Golang tiedot is a document database

Dec 11, 2022
Beerus-DB: a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic operations

Beerus-DB · Beerus-DB is a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic

Oct 29, 2022
OcppManager-go - A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables.

?? ocppManager-go A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables. Currently, only

Jan 3, 2022
Eventually consistent distributed in-memory cache Go library

bcache A Go Library to create distributed in-memory cache inside your app. Features LRU cache with configurable maximum keys Eventual Consistency sync

Dec 2, 2022
CockroachDB - the open source, cloud-native distributed SQL database.
CockroachDB - the open source, cloud-native distributed SQL database.

CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters. What is CockroachDB? Docs Quickstart C

Jan 2, 2023
A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

NutsDB English | 简体中文 NutsDB is a simple, fast, embeddable and persistent key/value store written in pure Go. It supports fully serializable transacti

Jan 1, 2023
The lightweight, distributed relational database built on SQLite.
The lightweight, distributed relational database built on SQLite.

rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it grace

Jan 5, 2023
TiDB is an open source distributed HTAP database compatible with the MySQL protocol
TiDB is an open source distributed HTAP database compatible with the MySQL protocol

Slack Channel Twitter: @PingCAP Reddit Mailing list: lists.tidb.io For support, please contact PingCAP What is TiDB? TiDB ("Ti" stands for Titanium) i

Jan 9, 2023
A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Vasto A distributed high-performance key-value store. On Disk. Eventual consistent. HA. Able to grow or shrink without service interruption. Vasto sca

Jan 6, 2023
A distributed MySQL binlog storage system built on Raft
A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Dec 31, 2022
Distributed cache with gossip peer membership enrollment.

Autocache Groupcache enhanced with memberlist for distributed peer discovery. TL;DR See /_example/ for usage. Run docker-compose -f _example/docker-co

Dec 8, 2022
LBADD: An experimental, distributed SQL database
LBADD: An experimental, distributed SQL database

LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat

Nov 29, 2022
TalariaDB is a distributed, highly available, and low latency time-series database for Presto
TalariaDB is a distributed, highly available, and low latency time-series database for Presto

TalariaDB is a distributed, highly available, and low latency time-series database that stores real-time data. It's built on top of Badger DB.

Nov 16, 2022
Distributed RWMutex in Go
Distributed RWMutex in Go

Distributed Read-Write Mutex in Go The default Go implementation of sync.RWMutex does not scale well to multiple cores, as all readers contend on the

Dec 29, 2022