500 Resources
Golang distributed-storage Libraries
A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.
DBPack DBPack means a database cluster tool pack. It can be deployed as a sidecar in a pod, it shields complex basic logic, so that business developme
Simple HTTP/HTTPS proxy - designed to be distributed as a self-contained binary that can be dropped in anywhere and run.
Simple Proxy This is a simple HTTP/HTTPS proxy - designed to be distributed as a self-contained binary that can be dropped in anywhere and run. Code b
Add, remove, and manage different versions of web-distributed software binaries. No elevated permissions required!
A cross-platform package manager for the web! Add, remove, and manage different versions of web-distributed software binaries. No elevated permissions
Kalasa is a NoSQL database and provides more data structures for ease of use.
Kalasa Kalasa is a NoSQL database and provides more data structures for ease of use. 简体中文 | English 特 性 嵌入的存储引擎 数据可以加密存储 可以自定义实现存储加密器 即使数据文件被拷贝,也保证存储数
A version control system to manage large files.
ArtiVC ArtiVC (Artifacts Version Control) is a handy command-line tool for data versioning on cloud storage. With only one command, it helps you neatl
Local Storage is one of HwameiStor components. It will provision the local LVM volume.
Local Storage Module English | Simplified_Chinese Introduction Local Storage is one of modules of HwameiStor which is a cloud native local storage sys
Ready to deploy, distributed cryptocurrency trading bot
HyperTrade Ready to deploy, distributed cryptocurrency trading bot. USE THIS SOFTWARE AT YOUR OWN RISK. THE AUTHOR ASSUMES NO LIABILITY FOR YOUR TRADI
MIT 6.824: Distributed Systems (Spring 2020)
MIT6.824 MIT 6.824: Distributed Systems (Spring 2020) Lab 1 Lab 2 Lab 2A Lab 2B Lab 2C Lab 2D Lab 3 Lab 3A Lab 3B Lab 4 Lab 4A Lab 4B Lab 4 Challenge
Grafana Mimir provides horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus.
Grafana Mimir Grafana Mimir is an open source software project that provides a scalable long-term storage for Prometheus. Some of the core strengths o
Tasqueue is a simple, lightweight distributed job/worker implementation in Go
Tasqueue Tasqueue is a simple, lightweight distributed job/worker implementation in Go Concepts tasqueue.Broker is a generic interface that provides m
Minicache - Distributed cache implemented in Go. Like Redis but simpler.
Distributed cache with client-side consistent hashing, distributed leader-elections, and dynamic node discovery. Supports both HTTP/gRPC interfaces secured with mTLS.
Go-redisson: A redisson like distributed redis lock, support watchdog、reentrant lock
go-redisson a Redisson like distributed locking implementation using Redis. Installation go get github.com/cheerego/go-redisson Lock Category Mutex Ex
ArcticDB - an embeddable columnar database written in Go
This project is still in its infancy, consider it not production-ready, probably has various consistency and correctness problems and all API will cha
Golang distributed Slowloris attack 🦥
slowloris - Golang distributed Slowloris attack How it works Read the article 🦷 How to protect from it TBD Installation Run go install github.com/its
Kitten is a distributed file system optimized for small file storage, inspired by Facebook's Haystack.
Kitten is a distributed file system optimized for small file storage, inspired by Facebook's Haystack.
Mini file storage with Go (Golang)
#Microstorage This is my exercise of creating simple file storage with GoLang Purpose: store and manipulate with user`s images in my pet projects ##Ve
Distributed Commit Log from Travis Jeffery's Distributed Services book
proglog A distributed commit log. This repository follows the book "Distributed Services with Go" by Travis Jeffrey. The official repository for this
azqlite is a lightweight wrapper around Azure's SDK to interact with the Azure Storage Queue service in a simpler and more idiomatic way.
azqlite azqlite is a lightweight wrapper around github.com/Azure/azure-storage-queue-go to interact with the Azure Storage Queue service in a simpler
Tapestry is an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs.
Tapestry This project implements Tapestry, an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate
Easy-to-use distributed multi-function current limiter.
go-limiter Easy-to-use distributed multi-function current limiter. Installation Run the following command under your project: go get -u github.com/NIC
Go-distributed-websocket - Distributed Web Socket with Golang and Redis
go-distributed-websocket Distributed Web Socket with Golang and Redis Dependenci
Verifier - Golang JWT token verifier with storage(default Redis)
verifier Golang JWT token verifier with storage(default Redis) Usage go get -u g
MyCache - A distributed cache based on GeeCache
借鉴GeeCache实现了MyCache(https://geektutu.com/post/geecache.html) 主要功能: 1.实现了fifo和lr
Jaeger-influxdb - The repository that contains InfluxDB Storage gRPC plugin for Jaeger
NOTICE: This repository is archived and is no longer maintained. Please use http
Users - Restful API that can get/create/update/delete user data from persistence storage
Users Service The users service is used to maintain Restful API that can get/cre
Querycrate - A simple library that loads and keeps SQL queries from files
QueryCrate A simple library for loading & getting string queries from files. How
Golang REST Layer SQL Storage Handler
This REST Layer resource storage backend stores data in a SQL Database using database/sql.
LotsusDB is a fast k/v database compatible with LSM tree and B+ tree
LotusDB LotsusDB is a fast k/v database compatible with LSM tree and B+ tree. Key features: Combine the advantages of LSM and B+ tree Fast read/write
Demonstrate a bounded context distributed over multiple repositories. In `go`
Contextive Demo - Go - Service This repository illustrates the use of Contextive in an environment where multiple repositories are part of the same bo
SMART information of local storage devices as Prometheus metrics
hpessa-exporter Overview Openshift's hpessa-exporter allows users to export SMART information of local storage devices as Prometheus metrics, by using
A Go implementation of an in-memory bloom filter, with support for boltdb and badgerdb as optional data persistent storage.
Sprout A bloom filter is a probabilistic data structure that is used to determine if an element is present in a set. Bloom filters are fast and space
Asset storage and on-the-fly image resize
air - Asset & Image Resize Asset storage and on-the-fly image resize powered by libvips. Uploading an asset $ http -f POST http://127.0.0.1:1323/uploa
Every 10 minutes, memory, cpu and storage usage is checked and if they over 80%, sending alert via email.
linux-alert Every 10 minutes, memory, cpu and storage usage is checked and if they over 80%, sending alert via email. Usage Create .env file from .env
BlobStore is a highly reliable,highly available and ultra-large scale distributed storage system
BlobStore Overview Documents Build BlobStore Deploy BlobStore Manage BlobStore License Overview BlobStore is a highly reliable,highly available and ul
Couchbase - distributed NoSQL cloud database
couchbase Couchbase is distributed NoSQL cloud database. create Scope CREATE SCO
Descentralized object storage network for resilient services
Polar Descentralized object storage network for resilient services Polar is a binary that you can run on any linux distribution, it will expose a RPC
Distributed-system - Practicing and learning the foundations of DS with Go
Distributed-System For practicing and learning the foundations of distributed sy
The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your container orchestrator
fortress-csi The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your co
Jaeger-s3 - Jaeger gRPC storage plugin for Amazon S3
jaeger-s3 jaeger-s3 is gRPC storage plugin for Jaeger, which uses Amazon Kinesis
Tinykv - The TinyKV course builds a key-value storage system with the Raft consensus algorithm
The TinyKV Course The TinyKV course builds a key-value storage system with the R
Log to cloud object storage for golang. Zap extension.
cos-logger Log to cloud object storage for golang implemented as io.Writer. Use it as a plugin/extension to uber-go/zap logger Configure logger and ad
A simple image hosting script in Golang (Smart Storage with Telegram Cloud Storage)
image-upload-tg It's a simple image hosting script in Golang. It's have http server and image can be uploaded from here, also images store temporary i
Openshift's hpessa-exporter allows users to export SMART information of local storage devices as Prometheus metrics, by using HPE Smart Storage Administrator tool
hpessa-exporter Overview Openshift's hpessa-exporter allows users to export SMART information of local storage devices as Prometheus metrics, by using
Labs for MIT 6.824 Distributed Systems (Spring 2020)
6.824-2020-labs This repo contains labs for MIT 6.824 Distributed Systems (Spring 2020) The master branch contains the source code with no implementat
Keyval - A simple key-value storage library written in Go
keyval keyval is a simple key-value storage library written in Go and its back c
Disgo - A distributed lock based on redis developed with golang
English | 中文 DisGo Introduce DisGo is a distributed lock based on Redis, develop
free4.chat is an instant audio conferencing service
free4chat free4.chat is an instant audio conferencing service. It is designed by the local first and privacy first principle, and is very easy to use.
ttlcache – is a simple and efficient in-memory key value storage with TTL for each record
ttlcache About ttlcache – is a simple and efficient in-memory key value storage with TTL for each record. The key is of uint64 type. Library provides
A letsencrypt client that uses etcd as its storage.
letsencrypt-with-etcd This is a letsencrypt client that uses etcd as its storage. It stores your (automatically created) LetsEncrypt account in /letse
This service is intented to collect data using grpc using Go lang backend and cassandra DB as storage
Go Data Collection This service is intented to collect data using grpc using Go lang backend and cassandra DB as storage. Dev Setup make test_env make
A Go distributed systems development framework
micro A Go distributed systems development framework framework eference resources base on go-micro support list grpc-balancer grpc-naming dns pool etc
A pure Golang implementation of Rockchip rknand vendor storage interface.
go-rkvendorstorage A pure Golang implementation of Rockchip rknand vendor storage interface. Usage package main import ( "fmt" "github.com/jamesits
A cross goroutine storage tool with very simple implementation and function.
Simple-goroutine-local is a cross goroutine storage tool with very simple implementation and function (the concept is similar to Java ThreadLocal). Ge
Topology-tester - Application to easily test microservice topologies and distributed tracing including K8s and Istio
Topology Tester The Topology Tester app allows you to quickly build a dynamic mi
Example of a distributed voting app running on Kubernetes. Written in Golang with Terraform definitions to deploy to AWS EKS
Example of a distributed voting app running on Kubernetes. Written in Golang with Terraform definitions to deploy to AWS EKS
hivefleet is a distributed, scalable load-testing tool built in go that leverages Google Cloud Functions
hivefleet is a distributed, scalable load-testing tool built in go that leverages Google Cloud Functions
Dsf - Singleflight for distributed senerios
dsf: Distributed SingleFlight Inspired by singleflight, this package provides a
Example of trace instrumentation in Golang applications :bar_chart:
go-opentelemetry-example Example of trace instrumentation in Golang applications using the opentelemetry. Requirements/dependencies Docker Docker-comp
Go Micro: a standard library for distributed systems development
Go Micro Go Micro is a standard library for distributed systems development. Ove
Simple and minimal image server capable of storing, resizing, converting and caching images.
webp-server Simple and minimal image server capable of storing, resizing, converting, and caching images. You can quickly find out how it works by loo
REST api using fiber framework written in golang and using firebase ecosystem to authentication, storage and firestore as a db and use clean architecture as base
Backend API Example FiberGo Framework Docs : https://github.com/gofiber Info This application using firebase ecosystem Firebase Auth Cloud Storage Fir
Uptrace - Distributed tracing backend using OpenTelemetry and ClickHouse
Distributed tracing backend using OpenTelemetry and ClickHouse Uptrace is a dist
Labs, solutions and related materials from the MIT 6.824 Distributed Systems course.
MIT 6.824 Distributed Systems Labs, solutions and related materials from the MIT 6.824 Distributed Systems course. Overview From the official website:
Imaginarium - A simple golang image storage engine
Imaginarium A simple golang image storage engine. Used to create and store diffe
Dxc - Go implementation of DxChain3.0 protocol
DxChain 3.0 The Ecosystem Powered by DxChain 3.0 Smart Contract Platform While c
Distributed-Log-Service - Distributed Log Service With Golang
Distributed Log Service This project is essentially a result of my attempt to un
Flash-metrics - Flash Metrics Storage With Golang
Flash Metrics Storage bootstrap: $ echo -e "max-index-length = 12288" tidb.con
Hive-fleet: a distributed, scalable load-testing tool built in go that leverages Google Cloud Functions
hive-fleet hive-fleet is a distributed, scalable load-testing tool, built on top
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service
Distributed-Services This project is essentially a result of my attempt to under
Colonies is a generic framework for implementing next-generation distributed applications and systems
Colonies is a generic framework for implementing next-generation distributed applications and systems. It can be used as a building block for grid computing or edge computing, e.g. implement a meta operating system or cloud-of-cloud platform that combines many execution environments into a new virtual computing environment that can be controlled using an single unified API.
Simple CLI interface to in-memory key/value storage a la redis.
Simple memory key value Simple CLI interface to in-memory key/value storage a la redis. Running Can be run through go directly or through docker using
Nightingale - A Distributed and High-Performance Monitoring System. Prometheus enterprise edition
Introduction 💡 A Distributed and High-Performance Monitoring System. Prometheus
Implementation of distributed key-value system based on TiKV
Distributed_key-value_system A naive implementation of distributed key-value system based on TiKV Features Features of this work are listed below: Dis
fleet ties together systemd and etcd into a distributed init system
Deprecation warning fleet is no longer developed or maintained by CoreOS. After February 1, 2018, a fleet container image will continue to be availabl
Code portion for Distributed System Final exam for Viktor Máni Mønster
DISYS Final Exam (Distributed Hash Table) How to run There are two components of the system, which need to be run in separate ways. As this is a distr
Go-storage - A vendor-neutral storage library for Golang: Write once, run on every storage service.
go-storage Website | Documentation | Community A vendor-neutral storage library for Golang. Vision Write once, run on every storage service. Goal Vend
Nano - Lightweight, facility, high performance golang based game server framework
Nano Nano is an easy to use, fast, lightweight game server networking library fo
OVO is an In-Memory Distributed Cache and a Key/Value Storage.
OVO is an In-Memory Distributed Cache and a Key/Value Storage. Main features OVO is a distributed in-memory cache that supports data sharding on m
Go-explosion - Distributed indexing and searching in Go/Golang
go-explosion Distributed indexing and searching in Go/Golang. This library does
Remaphore - Admin tool employing NATS to coordinate processes on distributed infrastructure.
remaphore Admin tool employing NATS to coordinate processes on distributed infrastructure. Tasks on widely distributed machines often have to be coord
Contentrouter - Protect static content via Firebase Hosting with Cloud Run and Google Cloud Storage
contentrouter A Cloud Run service to gate static content stored in Google Cloud
My solutions to labs of MIT 6.824: Distributed Systems.
MIT 6.824 Distributed Systems Labs
A collection about awesome blockchains
A collection about awesome blockchains - open distributed public databases w/ crypto hashes incl. git ;-). Blockchains are the new tulips :tulip::tulip::tulip:. Distributed is the new centralized.
Vulcan extends Prometheus adding horizontal scalability and long-term storage
Warning: This project is currently not maintained, and there is no plan to do so ATM. Vulcan Vulcan extends Prometheus adding horizontal scalability a
Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
Overview Thanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity, which can be added
M3 monorepo - Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Graphite Compatible, Metrics Platform
M3 Distributed TSDB and Query Engine, Prometheus Sidecar, Metrics Aggregator, and more such as Graphite storage and query engine. Table of Contents Mo
A time series storage engine for Go
catena n. A closely linked series. Catena is a storage engine for time series data. Documentation Extensive documentation is not available yet, but th
a Framework for creating microservices using technologies and design patterns of Erlang/OTP in Golang
Technologies and design patterns of Erlang/OTP have been proven over the years. Now in Golang. Up to x5 times faster than original Erlang/OTP in terms
Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.
Goka Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go. Goka aims to reduce the complexity of buildi
Transactional outbox harvester for Postgres → Kafka, written in Go
goharvest is a Go implementation of the Transactional Outbox pattern for Postgres and Kafka. While goharvest is a complex beast, the end result is dea
A Lightweight "Remote Cache Access" (Rekas) Framework
Rekas 项目介绍 Rekas:一个轻量级分布式缓存系统 框架 ,解决缓存系统中出现的缓存击穿[锁机制]、缓存穿透[布隆过滤器]、缓存雪崩[分布式]问题,实现
Distributed tracing using OpenTelemetry and ClickHouse
Distributed tracing backend using OpenTelemetry and ClickHouse Uptrace is a dist
A cross-language distributed transaction manager
English Docs 跨语言分布式事务管理器 DTM是一款golang开发的分布式事务管理器,解决了跨数据库、跨服务、跨语言栈更新数据的一致性问题。 他优雅
Go client for Apptainer key storage and retrieval using HKP
Container Key Client This project provides a Go client to Apptainer for key stor
Labs from MIT's graduate-level Distributed Systems course
Labs from MIT's graduate-level Distributed Systems course Course website here Lab 1: MapReduce Lab 2: Raft Consensus Algorithm Lab 2A: Raft Leader Ele
🚀A fast, stable and embedded k-v storage in pure Golang, supports string, list, hash, set, sorted set.
English | 简体中文 rosedb is not production-ready, we are doing more test now. rosedb is a fast, stable and embedded key-value (k-v) storage engine based
Secure SDK/vault for personal records/PII built to comply with GDPR
Databunker Databunker is a network-based, self-hosted, GDPR compliant, secure vault for personal data or PII: https://databunker.org/ Project demo is