Yet another Bloomfilter implementation in Go, compatible with Java's Guava library

bloomfilter

Build codecov Go Reference Go Report Card Mentioned in Awesome Go

Overview

Yet another Bloomfilter implementation in Go, compatible with Java's Guava library. This library borrows how Java's Guava libraray implements Bloomfilter hashing strategies to achieve the serialization compatibility.

Installing

First pull the latest version of the library:

go get github.com/OldPanda/bloomfilter

Then import the this library in your code:

import "github.com/OldPanda/bloomfilter"

Usage Examples

Basic Usage

package main

import (
	"fmt"

	"github.com/OldPanda/bloomfilter"
)

func main() {
	// create bloomfilter with expected insertion=500, error rate=0.01
	bf, _ := bloomfilter.NewBloomFilter(500, 0.01)
	// add number 0~199 into bloomfilter
	for i := 0; i < 200; i++ {
		bf.Put(i)
	}

	// check if number 100 and 200 are in bloomfilter
	fmt.Println(bf.MightContain(100))
	fmt.Println(bf.MightContain(200))
}

Serialization

package main

import "github.com/OldPanda/bloomfilter"

func main() {
	// expected insertion=500, error rate=0.01
	bf, _ := bloomfilter.NewBloomFilter(500, 0.01)
	// add 0~199 into bloomfilter
	for i := 0; i < 200; i++ {
		bf.Put(i)
	}

	// serialize bloomfilter to byte array
	bytes := bf.ToBytes()
	// handling the bytes ...
}

Deserialization

package main

import (
	"fmt"

	"github.com/OldPanda/bloomfilter"
)

func main() {
	// create bloomfilter from byte array
	bf, _ := bloomfilter.FromBytes(bytes)
	// check whether number 100 is in bloomfilter
	fmt.Println(bf.MightContain(100))
}

Benchmark

The benchmark testing runs on element insertion and query separately.

» go test -bench . -benchmem ./...
# github.com/OldPanda/bloomfilter.test
goos: darwin
goarch: amd64
pkg: github.com/OldPanda/bloomfilter
cpu: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
BenchmarkBloomfilterInsertion-8                  4923142               387.1 ns/op            17 B/op          1 allocs/op
BenchmarkBloomfilterQuery-8                      4678299               259.6 ns/op            15 B/op          1 allocs/op
BenchmarkBloomfilterDeserialization-8             162871              7110 ns/op           13200 B/op         52 allocs/op
PASS
ok      github.com/OldPanda/bloomfilter 4.880s
Owner
南村群童欺我老无力
null
Similar Resources

Go implementation of the geodesic routines from GeographicLib

Go implementation of the geodesic routines from GeographicLib

geodesic This package is a Go implementation of the geodesic routines from GeographicLib. Features Pure Go implementation Distance calculations with n

Dec 23, 2022

go-logr implementation with pterm

go-logr implementation with pterm

plogr go-logr implementation with pterm Usage See examples Add more colors and levels By default, only level 0 (info) and level 1 (debug) are supporte

Dec 22, 2021

Reference go implementation of globaldce protocol

globaldce-go This is the reference implementation of the command line interface of globaldce coded in the go programming language. This project is sti

Nov 8, 2021

Go implementation Welford’s method for one-pass variance computation

Welford - Online method of calculating variance and standard deviation Go implementation Welford’s method for one-pass variance computation with D. H.

Jan 6, 2023

Sliding window counters Redis rate limiting implementation for Golang

Sliding window counters Redis rate limiting implementation for Golang (Based on the Figma API rate limit algorithm)

Dec 21, 2022

Implementation for validating the NZ COVID Pass.

NZCP validator Validates NZCP passes according to https://nzcp.covid19.health.nz. Example See example_test.go and tests for more examples. func Exampl

Dec 20, 2021

Generic Free List implementation to reuse memory and avoid allocations

gofl GOFL provides a Generic Free List implementation for Go. Installation This

Oct 17, 2022

yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JSON Patch, directly transposed to YAML

yaml-patch yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JavaScript Object Notation (JSON) Patch, directly trans

Jan 15, 2022

A pure Golang implementation of Rockchip rknand vendor storage interface.

go-rkvendorstorage A pure Golang implementation of Rockchip rknand vendor storage interface. Usage package main import ( "fmt" "github.com/jamesits

Nov 8, 2022
Comments
  • runtime issue with garbage bloomfilter values

    runtime issue with garbage bloomfilter values

    this library will crash and fail if one creates a garbage bloomfilter value

    this is the byte array i used: [1,1]

    and this is the error i see

    panic: runtime error: index out of range [3] with length 0 goroutine 223 [running]: encoding/binary.bigEndian.Uint32(...) /usr/local/go/src/encoding/binary/binary.go:112 github.com/OldPanda/bloomfilter.FromBytes(0xc000132170, 0x2, 0x8, 0x1, 0x1, 0x10) /Users/odria/go/pkg/mod/github.com/!old!panda/[email protected]/bloomfilter.go:83 +0x613 main.isAcctAffiliated(0xc00081e0c0, 0x20, 0xc0006d41f8, 0x11, 0xc000430058, 0x8, 0x0) /Users/odria/dev/github/SAFE-ya-query-engine.v2/src/qe_succ_acct.go:65 +0x545 main.SummarizePage(0xc00025c000, 0x400a125, 0xc000803e00, 0x4, 0x4895bd0) /Users/odria/dev/github/SAFE-ya-query-engine.v2/src/qe_page_summary.go:293 +0x709 main.FetchHourlyFeatures(0xc00025c000, 0x4044316, 0x480e9f0, 0xc00009d620) /Users/odria/dev/github/SAFE-ya-query-engine.v2/src/qe_main.go:275 +0x45 main.FetchFeatures.func1(0xc0004300c0, 0xc00025c000, 0xc0005ae020, 0xc000314080, 0xc0005ae040) /Users/odria/dev/github/SAFE-ya-query-engine.v2/src/qe_main.go:195 +0x57 created by main.FetchFeatures /Users/odria/dev/github/SAFE-ya-query-engine.v2/src/qe_main.go:191 +0x16

EVM-compatible chain secured by the Lachesis consensus algorithm.

ICICB-Galaxy EVM-compatible chain secured by the Lachesis consensus algorithm. Building the source Building galaxy requires both a Go (version 1.14 or

Oct 7, 2021
safe and easy casting from one type to another in Go

cast Easy and safe casting from one type to another in Go Don’t Panic! ... Cast What is Cast? Cast is a library to convert between different go types

Jan 1, 2023
Utility to restrict which package is allowed to import another package.

go-import-rules Utility to restrict which package is allowed to import another package. This tool will read import-rules.yaml or import-rules.yml in t

Jan 7, 2022
go implementation of timsort

timsort timsort is a Go implementation of Tim Peters's mergesort sorting algorithm. For many input types it is 2-3 times faster than Go's built-in sor

Nov 7, 2022
Implementation of do255e and do255s in Go

Go Implementation of do255e and do255s This is a plain Go implementation of do255e and do255s. It is considered secure; all relevant functions should

Aug 15, 2022
Go implementation of the Heaven's Gate technique
Go implementation of the Heaven's Gate technique

gopherheaven is a Go implementation of the classic Heaven's Gate technique originally published by roy g biv on VX Heaven in 2009. gopherheaven can be used as an evasion technique to directly call 64-bit code from a 32-bit process.

Dec 20, 2022
Lightweight, Simple, Quick, Thread-Safe Golang Stack Implementation

stack Lightweight, Simple, Quick, Thread-Safe Golang Stack Implementation Purpose Provide a fast, thread safe, and generic Golang Stack API with minim

May 3, 2022
Optimal implementation of ordered maps for Golang - ie maps that remember the order in which keys were inserted.

Goland Ordered Maps Same as regular maps, but also remembers the order in which keys were inserted, akin to Python's collections.OrderedDicts. It offe

Jan 3, 2023
An idiomatic Go implementation of Leaky bucket.

lbucket lbucket is an idiomatic Go leaky bucket implementation. The library make use of plain old Go stdlib; in other words, there are no third-party

Apr 17, 2022
A faster RWLock primitive in Go, 2-3 times faster than RWMutex. A Go implementation of concurrency control algorithm in paper

Go Left Right Concurrency A Go implementation of the left-right concurrency control algorithm in paper <Left-Right - A Concurrency Control Technique w

Jan 6, 2023