MyCache - A distributed cache based on GeeCache

借鉴GeeCache实现了MyCache(https://geektutu.com/post/geecache.html)

主要功能:

1.实现了fifo和lru两种淘汰算法
2.基于一致性哈希支持分布式并发读写
3.使用了singleflight防止缓存击穿

结构:

consistenthash/consistenthash.go实现了一致性哈希,对外暴露的数据结构及方法包括:
// Map constains all hashed keys
type Map struct {
	hash     Hash           // Hash function
	replicas int            // Virtual node multiple
	keys     []int          // Sorted
	hashMap  map[int]string // Mapping Table of Virtual Node and Real Node
}

// New creates a Map instance
func New(replicas int, fn Hash) *Map 

// Add adds some keys to the hash.
func (m *Map) Add(keys ...string)

// Get gets the closest item in the hash to the provided key.
func (m *Map) Get(key string) string
eliminationstrategy/eliminationstrategy.go实现了fifo和lru淘汰算法的单机非并发cache,对外暴露的数据结构及方法包括:
// Cache is a LRU/FIFO cache. It is not safe for concurrent access.
type Cache struct {
	eliminationstrategy int
	maxBytes            int64
	nbytes              int64
	ll                  *list.List
	cache               map[string]*list.Element
	// optional and executed when an entry is purged.
	OnEvicted func(key string, value Value)
}

// New is the Constructor of Cache
func New(maxBytes int64, eliminationstrategy int, onEvicted func(string, Value)) *Cache

// Get look ups a key's value
func (c *Cache) Get(key string) (value Value, ok bool) 

// Add adds a value to the cache.
func (c *Cache) Add(key string, value Value)
singleflight/singleflight.go实现了只向远端节点发起一次请求以防止缓存击穿,对外暴露的数据结构及方法包括:
type Group struct {
	mu sync.Mutex // protects m
	m  map[string]*call
}

// Do only initiate one request to the same key
func (g *Group) Do(key string, fn func() (interface{}, error)) (interface{}, error)
byteview.go实现了 ByteView 用来表示缓存值,对外暴露的数据结构及方法包括:
// A ByteView holds an immutable view of bytes.
type ByteView struct {
	b []byte
}

// Len returns the view's length
func (v ByteView) Len() int

// ByteSlice returns a copy of the data as a byte slice.
func (v ByteView) ByteSlice() []byte

// String returns the data as a string, making a copy if necessary.
func (v ByteView) String() string 
cache.go为eliminationstrategy/eliminationstrategy.go中实现的cache添加了并发能力,对外暴露的数据结构及方法包括:
type cache struct {
	mu         sync.Mutex
	es         int
	esCache    *eliminationstrategy.Cache
	cacheBytes int64
}

// Add() includes New()
func (c *cache) Add(key string, value ByteView)

func (c *cache) Get(key string) (value ByteView, ok bool)
http.go提供了mycache的分布式http请求,peers.go提供了PeerPicker和PeerGetter接口

mycache.go组成了最终的分布式缓存,对外暴露的数据结构及方法包括:

// A Getter loads data for a key.
type Getter interface {
	Get(key string) ([]byte, error)
}

// A GetterFunc implements Getter with a function.
type GetterFunc func(key string) ([]byte, error)

// Get implements Getter interface function
func (f GetterFunc) Get(key string) ([]byte, error) {
return f(key)
}

// A Group is a cache namespace and associated data loaded spread over
type Group struct {
    name      string
    getter    Getter
    mainCache cache
    peers     PeerPicker
    // use singleflight.Group to make sure that each key is only fetched once
    loader *singleflight.Group
}

// NewGroup create a new instance of Group
func NewGroup(name string, es int, cacheBytes int64, getter Getter) *Group

// GetGroup returns the named group previously created with NewGroup, or
// nil if there's no such group.
func GetGroup(name string) *Group

// Get value for a key from cache
func (g *Group) Get(key string) (ByteView, error)

// RegisterPeers registers a PeerPicker for choosing remote peer
func (g *Group) RegisterPeers(peers PeerPicker)
Similar Resources

CockroachDB - the open source, cloud-native distributed SQL database.

CockroachDB - the open source, cloud-native distributed SQL database.

CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters. What is CockroachDB? Docs Quickstart C

Jan 2, 2023

The lightweight, distributed relational database built on SQLite.

The lightweight, distributed relational database built on SQLite.

rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it grace

Jan 5, 2023

TiDB is an open source distributed HTAP database compatible with the MySQL protocol

TiDB is an open source distributed HTAP database compatible with the MySQL protocol

Slack Channel Twitter: @PingCAP Reddit Mailing list: lists.tidb.io For support, please contact PingCAP What is TiDB? TiDB ("Ti" stands for Titanium) i

Jan 9, 2023

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Vasto A distributed high-performance key-value store. On Disk. Eventual consistent. HA. Able to grow or shrink without service interruption. Vasto sca

Jan 6, 2023

A distributed Configuration Center server that manages config in a container. The container is composed of fields (abstract layer includes: KV, LIST, DICT type). The Field contains basic datatypes (int, float, bool, string, list, dict).

A distributed Configuration Center server that manages config in a container. The container is composed of fields (abstract layer includes: KV, LIST, DICT type). The Field contains basic datatypes (int, float, bool, string, list, dict).

cassem config assembler from key-value pairs' container which include basic datatypes, such as int, string, float, bool, list, dict Features HTTP Rest

Nov 1, 2022

A distributed MySQL binlog storage system built on Raft

A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Dec 31, 2022

LBADD: An experimental, distributed SQL database

LBADD: An experimental, distributed SQL database

LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat

Nov 29, 2022

A course to build the SQL layer of a distributed database.

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

Jan 8, 2023

TalariaDB is a distributed, highly available, and low latency time-series database for Presto

TalariaDB is a distributed, highly available, and low latency time-series database for Presto

TalariaDB is a distributed, highly available, and low latency time-series database that stores real-time data. It's built on top of Badger DB.

Nov 16, 2022
Related tags
Distributed cache with gossip peer membership enrollment.

Autocache Groupcache enhanced with memberlist for distributed peer discovery. TL;DR See /_example/ for usage. Run docker-compose -f _example/docker-co

Dec 8, 2022
Distributed cache and in-memory key/value data store.

Distributed cache and in-memory key/value data store. It can be used both as an embedded Go library and as a language-independent service.

Dec 30, 2022
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order

Jan 9, 2023
IceFireDB - Distributed disk storage system based on Raft and RESP protocol.
IceFireDB - Distributed disk storage system based on Raft and RESP protocol.

Distributed disk storage database based on Raft and Redis protocol.

Dec 27, 2022
Disgo - A distributed lock based on redis developed with golang
Disgo - A distributed lock based on redis developed with golang

English | 中文 DisGo Introduce DisGo is a distributed lock based on Redis, develop

Dec 2, 2022
Efficient cache for gigabytes of data written in Go.

BigCache Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance. BigCache keeps entries on hea

Jan 4, 2023
:handbag: Cache arbitrary data with an expiration time.

cache Cache arbitrary data with an expiration time. Features 0 dependencies About 100 lines of code 100% test coverage Usage // New cache c := cache.N

Jan 5, 2023
Fast thread-safe inmemory cache for big number of entries in Go. Minimizes GC overhead

fastcache - fast thread-safe inmemory cache for big number of entries in Go Features Fast. Performance scales on multi-core CPUs. See benchmark result

Dec 30, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Dec 29, 2022
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache Summary groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many case

Dec 29, 2022