LevelDB key/value database in Go.

This is an implementation of the LevelDB key/value database in the Go programming language.

Build Status

Installation

go get github.com/syndtr/goleveldb/leveldb

Requirements

  • Need at least go1.5 or newer.

Usage

Create or open a database:

// The returned DB instance is safe for concurrent use. Which mean that all
// DB's methods may be called concurrently from multiple goroutine.
db, err := leveldb.OpenFile("path/to/db", nil)
...
defer db.Close()
...

Read or modify the database content:

// Remember that the contents of the returned slice should not be modified.
data, err := db.Get([]byte("key"), nil)
...
err = db.Put([]byte("key"), []byte("value"), nil)
...
err = db.Delete([]byte("key"), nil)
...

Iterate over database content:

iter := db.NewIterator(nil, nil)
for iter.Next() {
	// Remember that the contents of the returned slice should not be modified, and
	// only valid until the next call to Next.
	key := iter.Key()
	value := iter.Value()
	...
}
iter.Release()
err = iter.Error()
...

Seek-then-Iterate:

iter := db.NewIterator(nil, nil)
for ok := iter.Seek(key); ok; ok = iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Iterate over subset of database content:

iter := db.NewIterator(&util.Range{Start: []byte("foo"), Limit: []byte("xoo")}, nil)
for iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Iterate over subset of database content with a particular prefix:

iter := db.NewIterator(util.BytesPrefix([]byte("foo-")), nil)
for iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Batch writes:

batch := new(leveldb.Batch)
batch.Put([]byte("foo"), []byte("value"))
batch.Put([]byte("bar"), []byte("another value"))
batch.Delete([]byte("baz"))
err = db.Write(batch, nil)
...

Use bloom filter:

o := &opt.Options{
	Filter: filter.NewBloomFilter(10),
}
db, err := leveldb.OpenFile("path/to/db", o)
...
defer db.Close()
...

Documentation

You can read package documentation here.

Owner
Comments
  • Require explicit general database closing.

    Require explicit general database closing.

    To be idiomatic and play nicely as a library, control of the database should exist solely in the hands of the library user and not in that of the memory manager.

    Major Caveat: A number of internal types around iterators and versions retain their finalizers for now. I need to take some time to diagram their lifecycle to propose a solution for this. This domain is where, I believe, the nondeterminism I found was, but I want to start fixing the easy part first.

    @syndtr, if you have some recommendations for how to approach this, I would love to talk with you. If you look at other LevelDB implementations, they explicitly require the user to call Iterator.Close to free associated resources, including the an official but incomplete implementation from Google in Go at https://code.google.com/p/leveldb-go/source/browse/leveldb/db/db.go#72.

  • Slice bounds out of range error during compaction

    Slice bounds out of range error during compaction

    Here's the stackdump:

    panic: runtime error: slice bounds out of range [recovered] panic: runtime error: slice bounds out of range

    goroutine 50 [running]: github.com/syndtr/goleveldb/leveldb.func·015() /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:392 +0x172 github.com/syndtr/goleveldb/leveldb.iKey.ukey(0xc2006180df, 0x0, 0xa7, 0x0, 0xa7, ...) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/key.go:73 +0x75 github.com/syndtr/goleveldb/leveldb.(_iComparer).Compare(0xc20016a660, 0xc2006180df, 0x0, 0xa7, 0xc201207720, ...) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/comparer.go:21 +0x85 github.com/syndtr/goleveldb/leveldb/iterator.(_MergedIterator).smallest(0xc200afca50) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:178 +0x158 github.com/syndtr/goleveldb/leveldb/iterator.(_MergedIterator).Next(0xc200afca50, 0xc20061802d) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:113 +0x185 github.com/syndtr/goleveldb/leveldb.func·013(0x0, 0x0) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:247 +0x1cc github.com/syndtr/goleveldb/leveldb.(_DB).transact(0xc20012d180, 0x6c7a50) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:136 +0xaa github.com/syndtr/goleveldb/leveldb.(_DB).doCompaction(0xc20012d180, 0xc200172540, 0x6c7f00) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:365 +0x7e0 github.com/syndtr/goleveldb/leveldb.(_DB).compaction(0xc20012d180) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:463 +0x1cb created by github.com/syndtr/goleveldb/leveldb.open /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db.go:70 +0x1ec

  • Possible memory leak

    Possible memory leak

    I'm using goleveldb to keep an index from certain kinds metadata to entries in my logfiles. I have four leveldbs for each day which are rotated at midnight. After the process is running several days the memory consumption keeps growing and growing. I'm using a write buffer of 16 Mbytes.

    Here's the output from lsof:

    process 30008 user 68uW REG 253,2 0 15310869 ./20140527-indexdb.ldb/servicenames/LOCK process 30008 user 70w REG 253,2 165 15310870 ./20140527-indexdb.ldb/servicenames/LOG process 30008 user 74w REG 253,2 54 15310871 ./20140527-indexdb.ldb/servicenames/MANIFEST-000000 process 30008 user 82w REG 253,2 8740127 15310873 ./20140527-indexdb.ldb/servicenames/000001.log process 30008 user 129uW REG 253,2 0 15310875 ./20140527-indexdb.ldb/servicemetadata/LOCK process 30008 user 130w REG 253,2 165 15310876 ./20140527-indexdb.ldb/servicemetadata/LOG process 30008 user 131w REG 253,2 54 15310877 ./20140527-indexdb.ldb/servicemetadata/MANIFEST-000000 process 30008 user 134w REG 253,2 361647 15310879 ./20140527-indexdb.ldb/servicemetadata/000001.log process 30008 user 146uW REG 253,2 0 15310851 ./20140527-indexdb.ldb/base/LOCK process 30008 user 153w REG 253,2 24820721 15310912 ./20140527-indexdb.ldb/transactions/000002.log process 30008 user 155w REG 253,2 165 15310852 ./20140527-indexdb.ldb/base/LOG process 30008 user 159w REG 253,2 54 15310853 ./20140527-indexdb.ldb/base/MANIFEST-000000 process 30008 user 161w REG 253,2 59070 15310855 ./20140527-indexdb.ldb/base/000001.log process 30008 user 164uW REG 253,2 0 15310857 ./20140527-indexdb.ldb/transactions/LOCK process 30008 user 171w REG 253,2 365 15310858 ./20140527-indexdb.ldb/transactions/LOG process 30008 user 175w REG 253,2 128 15310859 ./20140527-indexdb.ldb/transactions/MANIFEST-000000 process 30008 user 192uW REG 253,2 0 15310863 ./20140527-indexdb.ldb/message/LOCK process 30008 user 193w REG 253,2 166 15310864 ./20140527-indexdb.ldb/message/LOG process 30008 user 194w REG 253,2 54 15310865 ./20140527-indexdb.ldb/message/MANIFEST-000000 process 30008 user 196w REG 253,2 26185266 15310867 ./20140527-indexdb.ldb/message/000001.log

    Here's the output from pprof:

    screenshot-1

  • Effort to reduce heap allocations

    Effort to reduce heap allocations

    Hi!

    I use goleveldb in syncthing (https://github.com/syncthing/syncthing) and have been doing a fair bit of profiling and hacking lately to reduce syncthing's memory footprint. I'm now at the point where goleveldb represents ~70-80 MB of a 100 MB footprint in my benchmarks. To a certain extent this is fair, with caches etc. But a large part of it is temporary heap allocations, in fact pretty much all of them made by snappy.Decode() from readRawBlock(). I made a naive attempt to reuse buffers with a sync.Pool in https://github.com/calmh/goleveldb/tree/reduce-heap, and it makes a dent but it's not huge. Probably the reason it's not very effective is that the pool isn't filled very often, and I'm not sure about the code paths that result in the data buffers becoming unused to hook into.

    This isn't a bug report as such, more a humble request to give it some work at some point, or pass me some hints and I'm happy to root around in it. :)

  • Compaction time can be quite big

    Compaction time can be quite big

    Hey! I am investigation the performance decline in our software https://github.com/tendermint/tendermint/issues/1835. We're using goleveldb as a primary DB for storing blocks plus some other data.

    Looks like saving a batch of data (~1MB) can result in ~ 15 sec. due to compaction process or something:

    https://github.com/tendermint/tendermint/issues/1835#issuecomment-402054099 (the time seems to be increasing with every spike)

    The write speed is approximately 1MB per second.

    Is this expected and we need to fine tune compaction options or this looks like a bug to you?

    Thank you!

  • Possible corruption on the block data

    Possible corruption on the block data

    This could be caused by bug in the table writer or block buffer.

    Corruption on the block buffer could be caused by same buffer acquired more than once from buffer pool by two different block instance (or probably added to buffer pool more than once?).

    https://gist.github.com/syndtr/a97bef48b12df8181d6b

    Recovering the DB doesn't indicate any corruption but the corrupted table may already been deleted.

    Could be related to syncthing/syncthing#987.

  • db corruption on 'out of disk space'

    db corruption on 'out of disk space'

    Using btcd, which currently uses goleveldb on commit 92de35cffa094e330ee854d44ccbcb954e344d08, if disk space runs out (write /tmp/mainnet/blocks_leveldb/000054.log: no space left on device), btcd can be restarted without problem.

    After updating goleveldb to commit 62cc4d4e8b1ba35c2d1597e54892c022c376995f, goleveldb corrupts the database on the same error. When btcd starts back up, it says leveldb: bad record in batch

  • Significant memory usage when trying to insert data

    Significant memory usage when trying to insert data

    I am trying to profile the memory usage of my go program, and found that the majority of memory seems to be consumed within leveldb functions (github.com/syndtr/goleveldb/leveldb/memdb.(*DB).Put).

    Below is the top profile output:

    (pprof) top
    6815712.39kB of 6881450.49kB total (99.04%)
    Dropped 372 nodes (cum <= 34407.25kB)
    Showing top 10 nodes out of 28 (cum >= 242528.01kB)
          flat  flat%   sum%        cum   cum%
     6471904kB 94.05% 94.05%  6471904kB 94.05%  github.com/syndtr/goleveldb/leveldb/memdb.(*DB).Put
    242528.01kB  3.52% 97.57% 242528.01kB  3.52%  bytes.makeSlice
       55040kB   0.8% 98.37%    55040kB   0.8%  lib.FP.GenerateHashMap
    46004.76kB  0.67% 99.04% 54040.69kB  0.79%  main.main
      149.16kB 0.0022% 99.04% 6483852.34kB 94.22%  github.com/syndtr/goleveldb/leveldb.(*DB).Write
       68.67kB 0.001% 99.04% 319400.85kB  4.64%  main.Worker
       13.77kB 0.0002% 99.04% 6507694.47kB 94.57%  main.Worker1
        4.03kB 5.9e-05% 99.04% 6495736.37kB 94.39%  github.com/syndtr/goleveldb/leveldb.(*DB).Put
             0     0% 99.04%   196616kB  2.86%  bytes.(*Buffer).WriteString
             0     0% 99.04% 242528.01kB  3.52%  bytes.(*Buffer).grow
    
  • disable auto compaction

    disable auto compaction

    Is there a way to disable the auto db compaction goleveldb does? My db keeps being corrupted, and if it does panic, it is always in db_compaction.go

    I want to test filling my DB without goleveldb running compaction at all to see if it still corrupts.

  • Compression change behind the scenes

    Compression change behind the scenes

    btcd (github.com/conformal/btcd) uses goleveldb to store data.

    The database is opened with:

            myCache := cache.NewEmptyCache()
            opts := &opt.Options{Flag: flag,
                    BlockCache:      myCache,
                    MaxOpenFiles:    256,
                    CompressionType: opt.NoCompression,
            }
    

    After a while, the database throws an error about a missing data.

    I added two fmt.Printfs to reader.go in readRawBlock()'s select case to print the type of Compression being used. I also added spew.Dump(bh) We have a tool to dump data.

    (table.blockHandle) {
     offset: (uint64) 1937402,
     length: (uint64) 4827
    }
    (uint8) 0
    No Compression
    

    On a different block:

    (table.blockHandle) {
     offset: (uint64) 0,
     length: (uint64) 22302
    }
    (uint8) 1
    Snappy Compression
    1381438195854721313 [Warn] unable to locate block sha 00000000000033640c6b34c40b8b95ba4fae759ec7970e546660a3abd9f9314c err snappy: unsupported COPY_4 tag
    1381438195854866024 [Info] Invalid block 00000000000033640c6b34c40b8b95ba4fae759ec7970e546660a3abd9f9314c
    

    And another:
    (table.blockHandle) {
     offset: (uint64) 1237318,
     length: (uint64) 3572
    }
    (uint8) 1
    Snappy Compression
    

    Yet another:

    (table.blockHandle) {
     offset: (uint64) 1119730,
     length: (uint64) 4331
    }
    (uint8) 0
    No Compression
    

    Any idea what is causing the compression to be changed? Multiple people are seeing this on different machines. I am the only one getting the COPY_4 tag issue.

  • go test fail on Windows/386

    go test fail on Windows/386

    Hello,

    I saw you fixed #14 so I wanted to test your package and here what I got:

    go version
    go version devel +d29da2ced72b Mon May 13 10:28:12 2013 +0400 windows/386
    

    Windows 7 x32

    Error:

    D:\Golibs\os\src\github.com\syndtr\goleveldb\leveldb>go test
    fatal error: runtime: cannot map pages in arena address space
    
    goroutine 328 [running]:
    [fp=0x16c787c] runtime.throw(0x665341)
            c:/Go/src/pkg/runtime/panic.c:473 +0x65
    [fp=0x16c789c] runtime.SysMap(0x3ffa0000, 0x100000)
            c:/Go/src/pkg/runtime/mem_windows.c:70 +0x78
    [fp=0x16c78bc] runtime.MHeap_SysAlloc(0x1730000, 0x100000)
            c:/Go/src/pkg/runtime/zmalloc_windows_386.c:360 +0x103
    [fp=0x16c78e4] MHeap_Grow(0x1730000, 0x10)
            c:/Go/src/pkg/runtime/mheap.c:224 +0x4f
    [fp=0x16c7904] MHeap_AllocLocked(0x1730000, 0x4, 0x2b)
            c:/Go/src/pkg/runtime/mheap.c:108 +0x286
    [fp=0x16c7918] runtime.MHeap_Alloc(0x1730000, 0x4, 0x2b, 0x0, 0x1, ...)
            c:/Go/src/pkg/runtime/mheap.c:77 +0x5c
    [fp=0x16c7950] MCentral_Grow(0x1b35d4c)
            c:/Go/src/pkg/runtime/mcentral.c:203 +0x81
    [fp=0x16c7974] runtime.MCentral_AllocList(0x1b35d4c, 0xc, 0x16c7988)
            c:/Go/src/pkg/runtime/mcentral.c:47 +0x4c
    [fp=0x16c7994] runtime.MCache_Alloc(0x15d0000, 0x2b, 0x1400, 0x1)
            c:/Go/src/pkg/runtime/mcache.c:24 +0x72
    [fp=0x16c79c4] runtime.mallocgc(0x1400, 0x1, 0x1, 0x1)
            c:/Go/src/pkg/runtime/zmalloc_windows_386.c:47 +0xdf
    [fp=0x16c79e4] makeslice1(0x5021e0, 0x13e0, 0x13e0, 0x16c7a18)
            c:/Go/src/pkg/runtime/slice.c:61 +0x7f
    [fp=0x16c7a04] runtime.makeslice(0x5021e0, 0x13e0, 0x0, 0x13e0, 0x0, ...)
            c:/Go/src/pkg/runtime/slice.c:34 +0x9c
    [fp=0x16c7a58] github.com/syndtr/goleveldb/leveldb/table.(*bInfo).readAll(0x3f9feb80, 0x201818c0, 0x3f9b4a98, 0x20181800, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/block.go:66 +0x74
    [fp=0x16c7a90] github.com/syndtr/goleveldb/leveldb/table.(*Reader).getBlock(0x3f9bd0f0, 0x3f9feb80, 0x20181440, 0x3f9b4988, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:171 +0x8e
    [fp=0x16c7ac8] github.com/syndtr/goleveldb/leveldb/table.(*Reader).getDataIter(0x3f9bd0f0, 0x3f9feb80, 0x20181440, 0x3f9b4988, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:202 +0x174
    [fp=0x16c7afc] github.com/syndtr/goleveldb/leveldb/table.(*indexIter).Get(0x3f9b19c0, 0x0, 0x0, 0x0, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:234 +0xd0
    [fp=0x16c7b14] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).setData(0x3f9b19e0, 0x1)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:138
    +0x2d
    [fp=0x16c7b20] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).Next(0x3f9b19e0, 0x3ff9d800)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:85 +0x7f
    [fp=0x16c7b2c] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).Next(0x3f9b17e0, 0x4ba7c0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:84 +0x45
    [fp=0x16c7b70] github.com/syndtr/goleveldb/leveldb/iterator.(*MergedIterator).Next(0x3ea83ea0, 0x3fdb03a0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:112 +0x124
    [fp=0x16c7c1c] github.com/syndtr/goleveldb/leveldb.func┬Ě015(0x0, 0x0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:247 +0x172
    [fp=0x16c7c6c] github.com/syndtr/goleveldb/leveldb.(*DB).transact(0x200bd500, 0x16c7cac)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:136 +0x74
    [fp=0x16c7f2c] github.com/syndtr/goleveldb/leveldb.(*DB).doCompaction(0x200bd500, 0x3e997480, 0x16c7f00)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:365 +0x75d
    [fp=0x16c7fd4] github.com/syndtr/goleveldb/leveldb.(*DB).compaction(0x200bd500)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:463 +0x17b
    [fp=0x16c7fd8] runtime.goexit()
            c:/Go/src/pkg/runtime/proc.c:1223
    created by github.com/syndtr/goleveldb/leveldb.openDB
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db.go:72 +0x1e6
    
    goroutine 1 [chan receive]:
    testing.RunTests(0x58aa7c, 0x666dc0, 0x3c, 0x3c, 0x1, ...)
            c:/Go/src/pkg/testing/testing.go:434 +0x69e
    testing.Main(0x58aa7c, 0x666dc0, 0x3c, 0x3c, 0x665a60, ...)
            c:/Go/src/pkg/testing/testing.go:365 +0x68
    main.main()
            C:/Users/Dobek/AppData/Local/Temp/go-build124087311/github.com/syndtr/goleveldb/leveldb/_test/_testmain.go:199 +0x80
    
    goroutine 327 [chan send]:
    github.com/syndtr/goleveldb/leveldb.(*DB).flush(0x200bd500, 0x0, 0x0, 0x0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:62 +0x176
    github.com/syndtr/goleveldb/leveldb.(*DB).Write(0x200bd500, 0x3ea85f20, 0x204cd010, 0x0, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:109 +0x181
    github.com/syndtr/goleveldb/leveldb.(*DB).Put(0x200bd500, 0x3ea6ecb0, 0xb, 0xb, 0x3eaa6400, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:162 +0x84
    github.com/syndtr/goleveldb/leveldb.(*dbHarness).put(0x204f5480, 0x3ea6eca0, 0xb, 0x21ba8800, 0x3e8, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_test.go:122 +0xb5
    github.com/syndtr/goleveldb/leveldb.TestDb_SparseMerge(0x2012b540)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_test.go:928 +0x165
    testing.tRunner(0x2012b540, 0x666f4c)
            c:/Go/src/pkg/testing/testing.go:353 +0x86
    created by testing.RunTests
            c:/Go/src/pkg/testing/testing.go:433 +0x683
    
    goroutine 329 [chan receive]:
    github.com/syndtr/goleveldb/leveldb.(*DB).writeJournal(0x200bd500)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:25 +0x3b
    created by github.com/syndtr/goleveldb/leveldb.openDB
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db.go:73 +0x1fb
    
    goroutine 7 [runnable]:
    exit status 2
    FAIL    github.com/syndtr/goleveldb/leveldb     15.997s
    

    Best regards, Dobrosław Żybort

  • slice bounds out of range [:18014398510996847] with capacity 4194304

    slice bounds out of range [:18014398510996847] with capacity 4194304

    I got a weird panic, what's going on?

    panic: runtime error: slice bounds out of range [:18014398510996847] with capacity 4194304 [recovered]
            panic: runtime error: slice bounds out of range [:18014398510996847] with capacity 4194304 [recovered]
            panic: runtime error: slice bounds out of range [:18014398510996847] with capacity 4194304
    goroutine 74586376 [running]:
    github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction.func1()
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:752 +0xc5
    panic({0xb58fc0, 0xc00e3dbda0})
            /usr/lib/go-1.18/src/runtime/panic.go:838 +0x207
    github.com/syndtr/goleveldb/leveldb.(*DB).compactionTransact.func1()
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:158 +0x9b
    panic({0xb58fc0, 0xc00e3dbda0})
            /usr/lib/go-1.18/src/runtime/panic.go:838 +0x207
    github.com/syndtr/goleveldb/leveldb/memdb.(*dbIter).fill(0x1e?, 0x91?, 0xe2?)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/memdb/memdb.go:42 +0x291
    github.com/syndtr/goleveldb/leveldb/memdb.(*dbIter).Next(0xc0166770e0?)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/memdb/memdb.go:131 +0x12e
    github.com/syndtr/goleveldb/leveldb.(*tOps).createFrom(0xc0031825a0?, {0xcab698, 0xc00c8a6080})
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/table.go:329 +0x106
    github.com/syndtr/goleveldb/leveldb.(*session).flushMemdb(0xc00807e4d0, 0xc01edb5900, 0xc012fb8000, 0x7faab3f5a498?)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/session_compaction.go:27 +0x105
    github.com/syndtr/goleveldb/leveldb.(*DB).memCompaction.func1(0xc016677001?)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:304 +0x9c
    github.com/syndtr/goleveldb/leveldb.(*compactionTransactFunc).run(0x3337333836300000?, 0x0?)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:241 +0x1f
    github.com/syndtr/goleveldb/leveldb.(*DB).compactionTransact(0xc00b0ec000, {0xb9f8b2, 0xb}, {0xca7fa0, 0xc00c896730})
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:185 +0x217
    github.com/syndtr/goleveldb/leveldb.(*DB).compactionTransactFunc(...)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:252
    github.com/syndtr/goleveldb/leveldb.(*DB).memCompaction(0xc00b0ec000)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:302 +0x3ee
    github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc00b0ec000)
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:766 +0x93
    created by github.com/syndtr/goleveldb/leveldb.openDB
            /root/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x5d6
    
  • some node will pass to f() function many times, is this a bug or a feature?

    some node will pass to f() function many times, is this a bug or a feature?

    https://github.com/syndtr/goleveldb/blob/126854af5e6d8295ef8e8bee3040dd8380ae72e8/leveldb/cache/cache.go#L327

            var nodes []*Node
    	for x := range h.buckets {
    		b := h.initBucket(uint32(x))
    
    		b.mu.Lock()
    		nodes = append(nodes, b.nodes...)
    		b.mu.Unlock()
    		f(nodes)
    	}
    
  • data race

    data race

    env GORACE=halt_on_error=1 go test -race ./...

    ==================
    WARNING: DATA RACE
    Read at 0x00c0001afd50 by goroutine 2253:
      runtime.chansend()
          /opt/hostedtoolcache/go/1.19.0/x64/src/runtime/chan.go:160 +0x0
      github.com/syndtr/goleveldb/leveldb.cAuto.ack()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_compaction.go:695 +0x87
      github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_compaction.go:787 +0xea
      github.com/syndtr/goleveldb/leveldb.openDB.func4()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:155 +0x39
    Previous write at 0x00c0001afd50 by goroutine 2248:
      runtime.closechan()
          /opt/hostedtoolcache/go/1.19.0/x64/src/runtime/chan.go:357 +0x0
      github.com/syndtr/goleveldb/leveldb.(*DB).compTriggerWait.func1()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_compaction.go:725 +0x39
      runtime.deferreturn()
          /opt/hostedtoolcache/go/1.19.0/x64/src/runtime/panic.go:476 +0x32
      github.com/syndtr/goleveldb/leveldb.(*DB).rotateMem()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_write.go:39 +0x75
      github.com/syndtr/goleveldb/leveldb.(*DB).flush.func1()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_write.go:106 +0x46a
      github.com/syndtr/goleveldb/leveldb.(*DB).flush()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_write.go:118 +0x2c1
      github.com/syndtr/goleveldb/leveldb.(*DB).writeLocked()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_write.go:157 +0xa6
      github.com/syndtr/goleveldb/leveldb.(*DB).Write()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_write.go:320 +0x92a
      github.com/syndtr/goleveldb/leveldb.TestDB_TransientError()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:2389 +0xe04
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_util.go:52 +0x1fe
      github.com/syndtr/goleveldb/leveldb.openDB()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:134 +0x7bd
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:489 +0xb2
      github.com/syndtr/goleveldb/leveldb.openDB()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:129 +0x7a7
      github.com/syndtr/goleveldb/leveldb.Open()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:202 +0x35c
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*session).recover.func1()
          /home/runner/work/goleveldb/goleveldb/leveldb/session.go:134 +0xd0
      runtime.deferreturn()
          /opt/hostedtoolcache/go/1.19.0/x64/src/runtime/panic.go:476 +0x32
      github.com/syndtr/goleveldb/leveldb.Open()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:188 +0x104
      github.com/syndtr/goleveldb/leveldb.(*dbHarness).openDB0()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:98 +0xb4
      github.com/syndtr/goleveldb/leveldb.(*dbHarness).init()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:89 +0x271
      github.com/syndtr/goleveldb/leveldb.newDbHarnessWopt()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:71 +0x167
      github.com/syndtr/goleveldb/leveldb.TestDB_TransientError()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:2360 +0x3e
      testing.tRunner()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1446 +0x216
      testing.(*T).Run.func1()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1493 +0x47
    Goroutine 2253 (running) created at:
      github.com/syndtr/goleveldb/leveldb.openDB()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:155 +0xa39
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*DB).checkAndCleanFiles()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_util.go:52 +0x1fe
      github.com/syndtr/goleveldb/leveldb.openDB()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:134 +0x7bd
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      fmt.Fscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:143 +0xe9
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:489 +0xb2
      github.com/syndtr/goleveldb/leveldb.openDB()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:129 +0x7a7
      github.com/syndtr/goleveldb/leveldb.Open()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:202 +0x35c
      fmt.Sscanf()
          /opt/hostedtoolcache/go/1.19.0/x64/src/fmt/scan.go:114 +0x164
      github.com/syndtr/goleveldb/leveldb/storage.fsParseName()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:651 +0x26
      github.com/syndtr/goleveldb/leveldb/storage.(*fileStorage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/storage/file_storage.go:465 +0x3e6
      github.com/syndtr/goleveldb/leveldb/testutil.(*Storage).List()
          /home/runner/work/goleveldb/goleveldb/leveldb/testutil/storage.go:349 +0x6d
      github.com/syndtr/goleveldb/leveldb.(*session).recover.func1()
          /home/runner/work/goleveldb/goleveldb/leveldb/session.go:134 +0xd0
      runtime.deferreturn()
          /opt/hostedtoolcache/go/1.19.0/x64/src/runtime/panic.go:476 +0x32
      github.com/syndtr/goleveldb/leveldb.Open()
          /home/runner/work/goleveldb/goleveldb/leveldb/db.go:188 +0x104
      github.com/syndtr/goleveldb/leveldb.(*dbHarness).openDB0()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:98 +0xb4
      github.com/syndtr/goleveldb/leveldb.(*dbHarness).init()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:89 +0x271
      github.com/syndtr/goleveldb/leveldb.newDbHarnessWopt()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:71 +0x167
      github.com/syndtr/goleveldb/leveldb.TestDB_TransientError()
          /home/runner/work/goleveldb/goleveldb/leveldb/db_test.go:2360 +0x3e
      testing.tRunner()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1446 +0x216
      testing.(*T).Run.func1()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1493 +0x47
    Goroutine 2248 (running) created at:
      testing.(*T).Run()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1493 +0x75d
      testing.runTests.func1()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1846 +0x99
      testing.tRunner()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1446 +0x216
      testing.runTests()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1844 +0x7ec
      testing.(*M).Run()
          /opt/hostedtoolcache/go/1.19.0/x64/src/testing/testing.go:1726 +0xa84
      main.main()
          _testmain.go:263 +0x2e9
    ==================
    FAIL	github.com/syndtr/goleveldb/leveldb	188.432s
    
  • iterator: optimize merged iterator (again)

    iterator: optimize merged iterator (again)

    • specialize index heap with int type
    • use heap fix instead of pop&push

    benchmarks:

    $ benchstat master.txt pr.txt 
    name                       old time/op  new time/op  delta
    MergedIterator/2_iters-8   8.97µs ± 0%  6.34µs ± 0%  -29.38%  (p=0.008 n=5+5)
    MergedIterator/50_iters-8   732µs ± 0%   580µs ± 1%  -20.67%  (p=0.008 n=5+5)
    
  • leveldb: reduce compaction table set of target level

    leveldb: reduce compaction table set of target level

    During table compaction, tables in source & target level are simply merged together and output to new tables. That's fine for a testing-purpose random dataset. But for a real case, sometimes none of keys in source level falls within some target level tables(L0->L1, especially) . Then these target level tables can be skipped.

    This PR implements this feature.

    Grepping the db log of my project with 'reducing' shows

    16:07:47.800017 [email protected] reducing L0 -> L1 F·-27 S·-86MiB
    16:07:57.902166 [email protected] reducing L0 -> L1 F·-7 S·-28MiB
    16:08:18.274260 [email protected] reducing L2 -> L3 F·-2 S·-819B
    16:08:19.743393 [email protected] reducing L2 -> L3 F·-4 S·-10KiB
    16:08:20.480615 [email protected] reducing L2 -> L3 F·-3 S·-1MiB
    
Golang-key-value-store - Key Value Store API Service with Go DDD Architecture

This document specifies the tools used in the Key-Value store and reorganizes how to use them. In this created service, In-Memory Key-Value Service was created and how to use the API is specified in the HTML file in the folder named "doc"

Jul 31, 2022
A key-value db api with multiple storage engines and key generation
A key-value db api with multiple storage engines and key generation

Jet is a deadly-simple key-value api. The main goals of this project are : Making a simple KV tool for our other projects. Learn tests writing and git

Apr 5, 2022
levigo is a Go wrapper for LevelDB

levigo levigo is a Go wrapper for LevelDB. The API has been godoc'ed and is available on the web. Questions answered at [email protected]egroups.com.

Aug 27, 2022
An embedded key/value database for Go.

Bolt Bolt is a pure Go key/value store inspired by Howard Chu's LMDB project. The goal of the project is to provide a simple, fast, and reliable datab

Sep 25, 2022
Key-value database stored in memory with option of persistence
Key-value database stored in memory with option of persistence

Easy and intuitive command line tool allows you to spin up a database avaliable from web or locally in a few seconds. Server can be run over a custom TCP protocol or over HTTP.

Aug 1, 2022
ZedisDB - a key-value memory database written in Go

ZedisDB - a key-value memory database written in Go

Sep 4, 2022
Simple Distributed key-value database (in-memory/disk) written with Golang.

Kallbaz DB Simple Distributed key-value store (in-memory/disk) written with Golang. Installation go get github.com/msam1r/kallbaz-db Usage API // Get

Jan 18, 2022
FlashDB is an embeddable, in-memory key/value database in Go
FlashDB is an embeddable, in-memory key/value database in Go

FlashDB is an embeddable, in-memory key/value database in Go (with Redis like commands and super easy to read)

Sep 18, 2022
A disk-backed key-value store.

What is diskv? Diskv (disk-vee) is a simple, persistent key-value store written in the Go language. It starts with an incredibly simple API for storin

Sep 14, 2022
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order

Sep 26, 2022
Distributed cache and in-memory key/value data store. It can be used both as an embedded Go library and as a language-independent service.

Olric Distributed cache and in-memory key/value data store. It can be used both as an embedded Go library and as a language-independent service. With

Sep 22, 2022
Simple, ordered, key-value persistence library for the Go Language

gkvlite gkvlite is a simple, ordered, ACID, key-value persistence library for Go. Overview gkvlite is a library that provides a simple key-value persi

Sep 3, 2022
Distributed, fault-tolerant key-value storage written in go.
Distributed, fault-tolerant key-value storage written in go.

A simple, distributed, fault-tolerant key-value storage inspired by Redis. It uses Raft protocotol as consensus algorithm. It supports the following data structures: String, Bitmap, Map, List.

Aug 4, 2022
a persistent real-time key-value store, with the same redis protocol with powerful features
a persistent real-time key-value store, with the same redis protocol with powerful features

a fast NoSQL DB, that uses the same RESP protocol and capable to store terabytes of data, also it integrates with your mobile/web apps to add real-time features, soon you can use it as a document store cause it should become a multi-model db. Redix is used in production, you can use it in your apps with no worries.

Sep 24, 2022
GhostDB is a distributed, in-memory, general purpose key-value data store that delivers microsecond performance at any scale.
GhostDB is a distributed, in-memory, general purpose key-value data store that delivers microsecond performance at any scale.

GhostDB is designed to speed up dynamic database or API driven websites by storing data in RAM in order to reduce the number of times an external data source such as a database or API must be read. GhostDB provides a very large hash table that is distributed across multiple machines and stores large numbers of key-value pairs within the hash table.

Aug 18, 2022
Pogreb is an embedded key-value store for read-heavy workloads written in Go.
Pogreb is an embedded key-value store for read-heavy workloads written in Go.

Embedded key-value store for read-heavy workloads written in Go

Sep 18, 2022
HA LDAP based key/value solution for projects configuration storing with multi master replication support

Recon is the simple solution for storing configs of you application. There are no specified instruments, no specified data protocols. For the full power of Recon you only need curl.

Jun 15, 2022
Fault tolerant, sharded key value storage written in GoLang
Fault tolerant, sharded key value storage written in GoLang

Ravel is a sharded, fault-tolerant key-value store built using BadgerDB and hashicorp/raft. You can shard your data across multiple clusters with mult

May 11, 2022
CrankDB is an ultra fast and very lightweight Key Value based Document Store.

CrankDB is a ultra fast, extreme lightweight Key Value based Document Store.

Apr 12, 2022