Lightweight RESTful database engine based on stack data structures

piladb Build Status GoDoc Go Report Card codecov osw

Logo

[pee-lah-dee-bee]. pila means stack or battery in Spanish.

piladb is a lightweight RESTful database engine based on stack data structures. Create as many stacks as you need, PUSH or POP elements of any kind, and have access to the one on top always in constant time.

Features

  • Stacks are auto-scalable and are only limited by the capacity of the host or by configuration.
  • Available POP, PUSH, PEEK,SIZE, and FLUSH operations for each of the stacks.
  • Manage stacks and other resources by using a REST API, so you can use it with your favorite programming language.
  • Manage elements in JSON-compatible data types: strings, numbers, arrays, objects, etc.
  • Totally configurable using a REST API, or CLI parameters.
  • In-memory store.
  • Written in Go, i.e. binaries are self-contained and distributable.

Documentation

Install

You can download binaries for Linux and Mac in the Releases page.

From Source Code

You need Go installed. Version 1.6+ recommended.

go get github.com/fern4lvarez/piladb/...
cd $GOPATH/src/github.com/fern4lvarez/piladb
make pilad

Clients

Development

You need Go installed. Version 1.6+ is mandatory.

go get github.com/fern4lvarez/piladb/...
cd $GOPATH/src/github.com/fern4lvarez/piladb
make all

You can also use Docker to create piladb builds or development environment. Please see the dev directory.

Dependencies

piladb aims to minimize the amount of third party dependencies and to rely on the Go standard library as much as possible.

Even though, it uses dep to vendor its few dependencies.

piladb also provides a go.mod file, which turns this project into a Go module. To learn more about Go modules and the vgo prototype, please read the Go & Versioning series by Russ Cox.

Code Coverage

We aim for a universal 100% code coverage for all suppackages. If some piece of code is not testable, it probably needs to be changed.

Check current code coverage of the project: https://codecov.io/gh/fern4lvarez/piladb

Release

You need Docker installed.

It's possible to get pilad binary releases by executing make release. This will cross-compile pilad in all available OS's and architectures.

Alternatively, if you don't have docker installed, you can release pilad binary with the make gox command. For this, you need a configured Go environment and gox installed.

Credits

piladb is developed by Fernando Álvarez and ≅oscillatingworks on a Dell XPS 13 laptop, running Ubuntu, and using vim-go plugin within the vim editor, in Berlin and Madrid, with the support of Gali, Godín and other friends.

Logo was designed by GraphicLoads.

Typography Lily Script One designed by Julia Petretta.

License

MIT

Owner
Fernando Álvarez
Ops @basecamp. OSS developer @oscillatingworks. Author of #piladb. Cities and Urbanism @UOC.
Fernando Álvarez
Comments
  • New Stack operations #44: FULL

    New Stack operations #44: FULL

    I am not really sure if putting isMaxStackSize in pilad/config.go is the best way, feel like check full should be encapsulate in the stack itself.

    I am open to suggestion.

    Description:
    
        Add a new rest endpoint FULL which checks if a particular stack is full or not.
    
    Testing Methodology:
    
        Verify the following curl call to full will return full = true
    
            ./bin/pilad -max-stack-size 0
            curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
            curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
            curl -XPOST localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF \
                -d '{"element":{"title":"1984","author":"George Orwell","ISBN":"1595404325","comments":[]}}'
            curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?full"
    
        Verify the following curl call to full will return full = false
    
            ./bin/pilad -max-stack-size 2
            curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
            curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
            curl -XPOST localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF \
                -d '{"element":{"title":"1984","author":"George Orwell","ISBN":"1595404325","comments":[]}}'
            curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?full"
    
  • New Stack operations #44: isEmpty

    New Stack operations #44: isEmpty

    Hi fern4lvarez,

    This pull request fixes the code reviews from https://github.com/fern4lvarez/piladb/pull/65

    1. Pull request merging to dev-0.2 instead of master
    2. change handler and api name to empty to conform naming convention
    3. Implement Empty function in stack.go
    4. Enforce 100% code test coverage.
    5. Allow maintainer to edit pull request.

    Description:

    Added isEmpty functionality to api. Now client can access is_empty rest call to
    determine if the stack associated to the pass in stack id and db id is empty or not.
    

    Testing Methodology:

    Verify the following curl call will return is_empty = false
    
        curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
        curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
        curl -XPOST localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF \
            -d '{"element":{"title":"1984","author":"George Orwell","ISBN":"1595404325","comments":[]}}'
        curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?is_empty"
    
    Verify the following curl call will return is_empty = true
    
            curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
            curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
            curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?is_empty"
    
    Verify that empty db or stack will have standard empty response
    
  • stack: fix data races

    stack: fix data races

    In particular, Peek could panic, but this doesn't prevent all race conditions (see commit msg).

    IMO, the stack should just embed a mutex so that the client can decide whether or not to acquire a lock.

  • New Stack operations #44: isEmpty

    New Stack operations #44: isEmpty

    Hi fern4lvarez,

    I am not sure if you are looking for contributors, but I saw your post on r/golang and got interested in the project. I saw issue New Stack operations #44 and decide to take a stab on one of the sub task. If you are cool with having contributors I will be happy to finish some of the other issues too.

    Description:

    Added isEmpty functionality to api. Now client can access is_empty rest call to
    determine if the stack associated to the pass in stack id and db id is empty or not.
    

    Testing Methodology:

    Verify the following curl call will return is_empty = false
    
        curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
        curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
        curl -XPOST localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF \
            -d '{"element":{"title":"1984","author":"George Orwell","ISBN":"1595404325","comments":[]}}'
        curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?is_empty"
    
    Verify the following curl call will return is_empty = true
    
            curl -XPUT "localhost:1205/databases?name=MY_DATABASE"
            curl -XPUT "localhost:1205/databases/MY_DATABASE/stacks?name=BOOKSHELF"
            curl "localhost:1205/databases/MY_DATABASE/stacks/BOOKSHELF?is_empty"
    
    Verify that empty db or stack will have standard empty response
    
  • Fix encoding bug when pushing a malformed payload

    Fix encoding bug when pushing a malformed payload

    Before this fix, if you tried to PUSH an element using some of the following payloads:

    • {"elent":8}
    • {"foo":8}
    • {1:8}

    The server wouldn't complain when parsing them, and would store a null on top of the Stack. Same behavior when trying to set some config value.

    With this fix, we now check that the payload contains the word element as key. Otherwise, we return an error when decoding it, that will become a 400 Bad Request in pilad.

  • pila: allow publicly to use custom Stack implementation

    pila: allow publicly to use custom Stack implementation

    This commit does not imply API breaking changes, so it can be added in the next 0.1.X release.

    Now it is possible to create a pila.Stack using a custom implementation that satisfies the stack.Stacker interface, e.g.

    type TestStack struct{}
    
    func (s *TestStack) Push(element interface{}) { return }
    func (s *TestStack) Pop() (interface{}, bool) { return nil, false }
    func (s *TestStack) Size() int                { return 0 }
    func (s *TestStack) Peek() interface{}        { return nil }
    func (s *TestStack) Flush()                   { return }
    
    stack := pila.NewStackWithBase("stack", time.Now(), &TestStack{})
    

    It is also possible to create a Stack with a custom base implementation from a Database instance:

    id := db.CreateStackWithBase("stack", time.Now(), &TestStack{})
    
  • Use UUID Version 5

    Use UUID Version 5

    We replace our old UUID implementation, which wasn't RFC 4122-compliant and use the Version 5 of https://github.com/satori/go.uuid, which is lightweight and well tested. This dependency is added to our vendor directory.

    This PR doesn't change the logic of piladb, only the way the Stacks and Databases UUIDs are represented.

  • Add option to allow pushing on a Stack when this is full

    Add option to allow pushing on a Stack when this is full

    Previously (0.1.X), when a Stack is full, PUSH operation is unavailable, returning 406 NOT ACCEPTABLE.

    This change provides a new config value, of boolean type:

    • PILADB_PUSH_WHEN_FULL as environment variable.
    • -push-when-full as CLI parameter.

    When this value is set to true, PUSH operations on a full Stack will be accepted, removing the bottommost element from the Stack.

    This PR also adds a new Sweep method to the Stack, which will remove the bottommost element from the Stack, returning such element and an OK if it was successfull. Note that the method is implemented at the Stack level, but there's no such SWEEP operation from the outside of a Pila.

  • Add support to codecov.io

    Add support to codecov.io

    Fixes #42

    • Add new script to execute tests on TravisCI gathering all coverage reports in one file.
    • Update TravisCi file in order to execute this script and send data to codecov.io.
    • Add codecov.io badge to README.
    • Update README regarding code coverage.
  • Introduce request for donations on startup

    Introduce request for donations on startup

    This can be disabled by using the -no-donate command line flag, or setting PILADB_NO_DONATE environment variable to true.

    This PR also introduces a new ASCII logo for the startup:

       .___.            _  _             _  _     
      /  _  \    _ __  (_)| |  __ _   __| || |__  
     |  |+|  |  | '_ \ | || | / _` | / _` || '_ \ 
     |  |-|  |  | |_) || || || (_| || (_| || |_) |
      \.___./   | .__/ |_||_| \__,_| \__,_||_.__/ 
                |_|   
    
  • Introduce graceful shutdown to pilad server

    Introduce graceful shutdown to pilad server

    Previously, when running a pilad server and then terminating it, it stopped immediately without caring about on-going connections.

    With this change, whenever pilad server receives SIGINT or SIGTERM signals, it will wait for on-going connections until they are finished. If the connections aren't finished after the Shutdown Timeout is due, they all will be interrupted and the shutdown will happen.

    This Shutdown Timeout can be configure with the pilad command with the -shutdown-timeout parameter, which value is 15 seconds by default. As other config values, it can be set on runtime using the /_config endpoint.

  • Add prod Dockerfile

    Add prod Dockerfile

    Add a prod Dockerfile for pilad with only 3 layers and weighting 11.2 MB. Also add DockerHub badge for /r/fern4lvarez/piladb in case of switching from dev image to prod image.

  • Implement version control

    Implement version control

    Version control will be the way piladb persists its contents: instead of writing data into disk, it will decode all atomic operations into a binary log, that will contain the history of the piladb server. This will make possible to recover the state of a piladb instance from a file in order to implement #14, and also to share the state with other instances in a distributed fashion, so we can achieve #13.

  • Write `piladb` complete documentation

    Write `piladb` complete documentation

    piladb needs proper documentation.

    The chosen approach is to use GitBook, as it provides Git/Github compatibility, Markdown, well structured sections, nice user interface and possibility to export to PDF, Mobi or ePub.

    The project will be called docs.piladb.org.

    GitBook site: https://www.gitbook.com/book/fern4lvarez/docs.piladb.org Github repository: https://github.com/oscillatingworks/docs.piladb.org

Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures.

Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures. capabilities which owl provides include Process approval、sql Audit、sql execute and execute as crontab、data backup and recover .

Nov 9, 2022
Hard Disk Database based on a former database

Hard Disk Database based on a former database

Nov 1, 2021
The lightweight, distributed relational database built on SQLite.
The lightweight, distributed relational database built on SQLite.

rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it grace

Jan 5, 2023
A lightweight document-oriented NoSQL database written in pure Golang.
A lightweight document-oriented NoSQL database written in pure Golang.

Lightweight document-oriented NoSQL Database ???? English | ???? 简体中文 | ???? Spanish CloverDB is a lightweight NoSQL database designed for being simpl

Jan 1, 2023
A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

go-mysql-server is a SQL engine which parses standard SQL (based on MySQL syntax) and executes queries on data sources of your choice. A simple in-memory database and table implementation are provided, and you can query any data source you want by implementing a few interfaces.

Dec 27, 2022
Fast Database engine in Go.

gaeadb gaeadb is a pure Go Database engine designed by nnsgmsone. The goal of the project is to provide a database engine for table or other complex d

Oct 14, 2022
Terraform provider for SQLite database engine

Terraform provider for SQLite database engine !!! WARNING !!! This is an educational project. Not intended for any production use! !!! WARNING !!! Her

Jun 11, 2022
This is a simple graph database in SQLite, inspired by "SQLite as a document database".

About This is a simple graph database in SQLite, inspired by "SQLite as a document database". Structure The schema consists of just two structures: No

Jan 3, 2023
Simple key value database that use json files to store the database

KValDB Simple key value database that use json files to store the database, the key and the respective value. This simple database have two gRPC metho

Nov 13, 2021
Beerus-DB: a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic operations

Beerus-DB · Beerus-DB is a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic

Oct 29, 2022
A RESTful caching micro-service in Go backed by Couchbase

Couchcache A caching service developed in Go. It provides REST APIs to access key-value pairs stored in Couchbase. You may also consider using couchca

Sep 26, 2022
Membin is an in-memory database that can be stored on disk. Data model smiliar to key-value but values store as JSON byte array.

Membin Docs | Contributing | License What is Membin? The Membin database system is in-memory database smiliar to key-value databases, target to effici

Jun 3, 2021
EliasDB a graph-based database.
EliasDB a graph-based database.

EliasDB EliasDB is a graph-based database which aims to provide a lightweight solution for projects which want to store their data as a graph. Feature

Jan 4, 2023
Time Series Database based on Cassandra with Prometheus remote read/write support

SquirrelDB SquirrelDB is a scalable high-available timeseries database (TSDB) compatible with Prometheus remote storage. SquirrelDB store data in Cass

Oct 20, 2022
rosedb is an embedded and fast k-v database based on LSM + WAL
rosedb is an embedded and fast k-v database based on LSM + WAL

A simple k-v database in pure Golang, supports string, list, hash, set, sorted set.

Dec 30, 2022
Rk-db - Enterprise level database bootstrapper with YAML based on rk-entry in Golang

rk-db Enterprise level database bootstrapper with YAML in golang. This belongs to rk-boot family. We suggest use this lib from rk-boot. Database Statu

Nov 16, 2022
A GPU-powered real-time analytics storage and query engine.
A GPU-powered real-time analytics storage and query engine.

AresDB AresDB is a GPU-powered real-time analytics storage and query engine. It features low query latency, high data freshness and highly efficient i

Jan 7, 2023
An embedded key/value database for Go.

bbolt bbolt is a fork of Ben Johnson's Bolt key/value store. The purpose of this fork is to provide the Go community with an active maintenance and de

Jan 1, 2023
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support

BuntDB is a low-level, in-memory, key/value store in pure Go. It persists to disk, is ACID compliant, and uses locking for multiple readers and a sing

Dec 30, 2022