321 Resources
Golang distributed Libraries
Centralized Configuration System written in Golang - Spring cloud compatible
Centralized Configuration System What is Vecosy Vecosy is a configuration service exposed through REST/GRPC. Is Spring Cloud Conf compatible and also
Distributed simple and robust release management and monitoring system.
Agente Distributed simple and robust release management and monitoring system. **This project on going work. Road map Core system First worker agent M
GoVector is a vector clock logging library written in Go.
GoVector is a vector clock logging library written in Go. The vector clock algorithm is used to order events in distributed systems in the absence of a centralized clock. GoVector implements the vector clock algorithm and provides feature-rich logging and encoding infrastructure.
LinDB is an open-source Time Series Database which provides high performance, high availability and horizontal scalability.
LinDB is an open-source Time Series Database which provides high performance, high availability and horizontal scalability. LinDB stores all monitoring data of ELEME Inc, there is 88TB incremental writes per day and 2.7PB total raw data.
Inspr is an application mesh for simple, fast and secure development of distributed applications.
Inspr is an engine for running distributed applications, using multiple communication patterns such as pub sub and more, focused on type consistency a
Vald. A Highly Scalable Distributed Vector Search Engine
Vald is a highly scalable distributed fast approximate nearest neighbor dense vector search engine.
Asynq: simple, reliable, and efficient distributed task queue in Go
Asynq: simple, reliable, and efficient distributed task queue in Go
Golang implementation of Sliding Window Algorithm for distributed rate limiting.
slidingwindow Golang implementation of Sliding Window Algorithm for distributed rate limiting. Installation $ go get -u github.com/RussellLuo/slidingw
Bigmachine is a library for self-managing serverless computing in Go
Bigmachine Bigmachine is a toolkit for building self-managing serverless applications in Go. Bigmachine provides an API that lets a driver process for
分布式定时任务库 distributed-cron
dcron 分布式定时任务库 原理 基于redis同步节点数据,模拟服务注册。然后将任务名 根据一致性hash 选举出执行该任务的节点。 流程图 特性 负载均衡:根据任务数据和节点数据均衡分发任务。 无缝扩容:如果任务节点负载过大,直接启动新的服务器后部分任务会自动迁移至新服务实现无缝扩容。
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is an open-source POSIX file system built on top of Redis and object storage
a Framework for creating mesh networks using technologies and design patterns of Erlang/OTP in Golang
Ergo Framework Implementation of Erlang/OTP in Golang. Up to x5 times faster than original Erlang/OTP. The easiest drop-in replacement for your hot no
go mysql driver, support distributed transaction
Go-MySQL-Driver A MySQL-Driver for Go's database/sql package Features Requirements Installation Usage DSN (Data Source Name) Password Protocol Address
Easy to use distributed event bus similar to Kafka
chukcha Easy to use distributed event bus similar to Kafka. The event bus is designed to be used as a persistent intermediate storage buffer for any k
A course to build distributed key-value service based on TiKV model
The TinyKV Course This is a series of projects on a key-value storage system built with the Raft consensus algorithm
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker
Celery Distributed Task Queue in Go
gocelery Go Client/Server for Celery Distributed Task Queue Why? Having been involved in several projects migrating servers from Python to Go, I have
a microservice framework for rapid development of micro services in Go with rich eco-system
中文版README Go-Chassis is a microservice framework for rapid development of microservices in Go. it focus on helping developer to deliver cloud native a
Fast Raft framework using the Redis protocol for Go
This project has been archived. Please check out Uhaha for a fitter, happier, more productive Raft framework. Finn is a fast and simple framework for
A feature complete and high performance multi-group Raft library in Go.
Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03
A distributed, fault-tolerant pipeline for observability data
Table of Contents What Is Veneur? Use Case See Also Status Features Vendor And Backend Agnostic Modern Metrics Format (Or Others!) Global Aggregation
CNCF Jaeger, a Distributed Tracing Platform
Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do
A course to build the SQL layer of a distributed database.
TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed
VectorSQL is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse.
NOTICE: This project have moved to fuse-query VectorSQL is a free analytics DBMS for IoT & Big Data, compatible with ClickHouse. Features High Perform
LBADD: An experimental, distributed SQL database
LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat
RadonDB is an open source, cloud-native MySQL database for building global, scalable cloud services
OverView RadonDB is an open source, Cloud-native MySQL database for unlimited scalability and performance. What is RadonDB? RadonDB is a cloud-native
Distributed reliable key-value store for the most critical data of a distributed system
etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order
Pholcus is a distributed high-concurrency crawler software written in pure golang
Pholcus Pholcus(幽灵蛛)是一款纯 Go 语言编写的支持分布式的高并发爬虫软件,仅用于编程学习与研究。 它支持单机、服务端、客户端三种运行模式,拥有Web、GUI、命令行三种操作界面;规则简单灵活、批量任务并发、输出方式丰富(mysql/mongodb/kafka/csv/excel等
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
Crawlab 中文 | English Installation | Run | Screenshot | Architecture | Integration | Compare | Community & Sponsorship | CHANGELOG | Disclaimer Golang-
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).
中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不
SeaweedFS is a distributed storage system for blobs, objects, files, and data warehouse, to store and serve billions of files fast! Blob store has O(1) disk seek, local tiering, cloud tiering. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.
SeaweedFS Sponsor SeaweedFS via Patreon SeaweedFS is an independent Apache-licensed open source project with its ongoing development made possible ent
High-Performance server for NATS, the cloud native messaging system.
NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo
A realtime distributed messaging platform
Source: https://github.com/nsqio/nsq Issues: https://github.com/nsqio/nsq/issues Mailing List: [email protected] IRC: #nsq on freenode Docs:
Distributed cache with gossip peer membership enrollment.
Autocache Groupcache enhanced with memberlist for distributed peer discovery. TL;DR See /_example/ for usage. Run docker-compose -f _example/docker-co
🦖 Streaming-Serverless Framework for Low-latency Edge Computing applications, running atop QUIC protocol, engaging 5G technology.
YoMo YoMo is an open-source Streaming Serverless Framework for building Low-latency Edge Computing applications. Built atop QUIC Transport Protocol an
Temporal service and CLI
Temporal Temporal is a microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7天用Go动手写/从零实现系列
7 days golang programs from scratch README 中文版本 7天用Go从零实现系列 7天能写什么呢?类似 gin 的 web 框架?类似 groupcache 的分布式缓存?或者一个简单的 Python 解释器?希望这个仓库能给你答案
Distributed Task Scheduling System|分布式定时任务调度平台
Crocodile Distributed Task Scheduling System English | 中文 Introduction A distributed task scheduling system based on Golang that supports http request
A Golang based high performance, scalable and distributed workflow framework
Go-Flow A Golang based high performance, scalable and distributed workflow framework It allows to programmatically author distributed workflow as Dire
A Distributed Continuous Integration System from MongoDB
Evergreen Evergreen is a distributed continuous integration system built by MongoDB. It dynamically allocates hosts to run tasks in parallel across ma
A programmable, observable and distributed job orchestration system.
📖 Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac
Open source Observability Platform. 👉 SigNoz helps developers find issues in their deployed applications & solve them quickly
SigNoz SigNoz is an opensource observability platform. SigNoz uses distributed tracing to gain visibility into your systems and powers data using Kafk
LBADD: An experimental, distributed SQL database
LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat
A distributed system for embedding-based retrieval
Overview Vearch is a scalable distributed system for efficient similarity search of deep learning vectors. Architecture Data Model space, documents, v
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
Cadence Visit cadenceworkflow.io to learn about Cadence. This repo contains the source code of the Cadence server. To implement workflows, activities
GO2P is a P2P framework, designed with flexibility and simplicity in mind
go2p golang p2p framework By v-braun - viktor-braun.de. Description GO2P is a P2P framework, designed with flexibility and simplicity in mind. You can
Akutan is a distributed knowledge graph store, sometimes called an RDF store or a triple store.
Akutan is a distributed knowledge graph store, sometimes called an RDF store or a triple store. Knowledge graphs are suitable for modeling data that is highly interconnected by many types of relationships, like encyclopedic information about the world. A knowledge graph store enables rich queries on its data, which can be used to power real-time interfaces, to complement machine learning applications, and to make sense of new, unstructured information in the context of the existing knowledge.
A distributed MySQL binlog storage system built on Raft
What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl
Visualize how a projects source code is distributed among its files and folders
Source Code Visualizer Visualize the code distribution in a project. Applications Applications include: Visualizing code distribution for more educate
Juno 译名朱诺。这个名字来源于古罗马神话中的众神之母。它是斗鱼的微服务管理系统, 如同众神之母一样守护着所有微服务的系统。
JUNO - A distributed application management system Introduction JUNO(朱诺) 是斗鱼数据服务组研发的分布式服务管理系统,核心功能为配置中心,能集中化管理应用不同环境、不同机房的配置;并且集成了监控中心、测试平台、日志中心等功能模块,
open source training courses about distributed database and distributed systemes
Welcome to learn Talent Plan Courses! Talent Plan is an open source training program initiated by PingCAP. It aims to create or combine some open sour
Distributed WebSocket broker
dSock dSock is a distributed WebSocket broker (in Go, using Redis). Clients can authenticate & connect, and you can send text/binary message as an API
Distributed RTC System by pure Go and Flutter
ION is a distributed real-time communication system, the goal is to chat anydevice, anytime, anywhere! Distributed Real-time Communication System ION
Implementation of the NELI leader election protocol for Go and Kafka
goNELI Implementation of the NELI leader election protocol for Go and Kafka. goNELI encapsulates the 'fast' variation of the protocol, running in excl
Lockgate is a cross-platform locking library for Go with distributed locks using Kubernetes or lockgate HTTP lock server as well as the OS file locks support.
Lockgate Lockgate is a locking library for Go. Classical interface: 2 types of locks: shared and exclusive; 2 modes of locking: blocking and non-block
A distributed Configuration Center server that manages config in a container. The container is composed of fields (abstract layer includes: KV, LIST, DICT type). The Field contains basic datatypes (int, float, bool, string, list, dict).
cassem config assembler from key-value pairs' container which include basic datatypes, such as int, string, float, bool, list, dict Features HTTP Rest
a distributed chat system which can be used as chat rooms or state synchronization
unitalk is a distributed chat system which can be used as chat rooms or state synchronization. unitalk registers itself on zookeeper when it start up, use redis cluster to broadcast messages or state, and permanent messages or state to kafka and then save to db.
CRAQ (Chain Replication with Apportioned Queries) in Go
CRAQ is a replication protocol that allows reads from any replica while still maintaining strong consistency. CRAQ should provide better read throughput than Raft and Paxos. Read performance grows linearly with the number of nodes added to the system. Network chatter is significantly lower compared to Raft and Paxos.
A distributed, proof of stake blockchain designed for the financial services industry.
Provenance Blockchain Provenance is a distributed, proof of stake blockchain designed for the financial services industry.
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.
Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.
distributed monitoring system
OWL OWL 是由国内领先的第三方数据智能服务商 TalkingData 开源的一款企业级分布式监控告警系统,目前由 Tech Operation Team 持续开发更新维护。 OWL 后台组件全部使用 Go 语言开发,Go 语言是 Google 开发的一种静态强类型、编译型、并发型,并具有垃圾回
Distributed, lock-free, self-hosted health checks and status pages
Checkup is distributed, lock-free, self-hosted health checks and status pages, written in Go. It features an elegant, minimalistic CLI and an idiomati
Micro is a platform for cloud native development
Micro Overview Micro addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provid
CNCF Jaeger, a Distributed Tracing Platform
Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do
GhostDB is a distributed, in-memory, general purpose key-value data store that delivers microsecond performance at any scale.
GhostDB is designed to speed up dynamic database or API driven websites by storing data in RAM in order to reduce the number of times an external data source such as a database or API must be read. GhostDB provides a very large hash table that is distributed across multiple machines and stores large numbers of key-value pairs within the hash table.
Distributed, fault-tolerant key-value storage written in go.
A simple, distributed, fault-tolerant key-value storage inspired by Redis. It uses Raft protocotol as consensus algorithm. It supports the following data structures: String, Bitmap, Map, List.
Go Open Source, Distributed, Simple and efficient Search Engine
Go Open Source, Distributed, Simple and efficient full text search engine.
A distributed unique ID generator inspired by Twitter's Snowflake
Sonyflake is a distributed unique ID generator inspired by Twitter's Snowflake.
Web-based Cloud Gaming service for Retro Game
CloudRetro provides an open-source cloud gaming platform for retro games. It started as an experiment for testing cloud gaming performance with WebRTC and libretro, and now it aims to deliver the most modern and convenient gaming experience through the technology.
NewSQL distributed storage database based on micro service framework
QLite 是基于微服务的 NewSQL 型数据库系统,与传统的一体化数据库不同,该系统将本该内置的多种数据结构(STL)拆分成多个服务模块,每个模块都是独立的一个节点,每个节点都与其主网关进行连接,从而形成分布式存储结构。
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is an open-source POSIX file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environ
Collyzar - A distributed redis-based framework for colly.
Collyzar A distributed redis-based framework for colly. Collyzar provides a very simple configuration and tools to implement distributed crawling/scra
⟁ Tendermint Core (BFT Consensus) in Go
Tendermint Byzantine-Fault Tolerant State Machines. Or Blockchain, for short. Branch Tests Coverage Linting master Tendermint Core is Byzantine Fault
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Goribot 一个分布式友好的轻量的 Golang 爬虫框架。 完整文档 | Document !! Warning !! Goribot 已经被迁移到 Gospider|github.com/zhshch2002/gospider。修复了一些调度问题并分离了网络请求部分到另一个仓库。此仓库会继续
Distributed reliable key-value store for the most critical data of a distributed system
etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order
Instant Kubernetes-Native Application Observability
What is Pixie? Pixie gives you instant visibility by giving access to metrics, events, traces and logs without changing code. We're building up Pixie
CNCF Jaeger, a Distributed Tracing Platform
Jaeger - a Distributed Tracing System Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing platform created by Uber Technologies and do
An enhanced HTTP client for Go
Heimdall Description Installation Usage Making a simple GET request Creating a hystrix-like circuit breaker Creating a hystrix-like circuit breaker wi
A distributed unique ID generator of using Sonyflake and encoded by Base58
Indigo About A distributed unique ID generator of using Sonyflake and encoded by Base58. ID max length is 11 characters by unsigned int64 max value. A
Go Micro is a framework for distributed systems development
Go Micro Go Micro is a framework for distributed systems development. Overview Go Micro provides the core requirements for distributed systems develop
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker
Asynq: simple, reliable, and efficient distributed task queue in Go
Asynq Overview Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be sc
Distributed hyperparameter optimization framework, inspired by Optuna.
Goptuna Distributed hyperparameter optimization framework, inspired by Optuna [1]. This library is particularly designed for machine learning, but eve
A consistent distributed data store.
Doozer What Is It? Doozer is a highly-available, completely consistent store for small amounts of extremely important data. When the data changes, it
Go language interface to the Libcircle distributed-queue API
Circle Description The Circle package provides a Go interface to the Libcircle distributed-queue API. Despite the name, Circle has nothing to do with
A cloud native distributed streaming network telemetry.
Panoptes Streaming Panoptes Streaming is a cloud native distributed streaming network telemetry. It can be installed as a single binary or clustered n
Industrial IoT Messaging and Device Management Platform
Mainflux Mainflux is modern, scalable, secure, open-source, and patent-free IoT cloud platform written in Go. It accepts user and thing (sensor, actua
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
glow Purpose Glow is providing a library to easily compute in parallel threads or distributed to clusters of machines. This is written in pure Go. I a
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built
Lightweight, facility, high performance golang based game server framework
Nano Nano is an easy to use, fast, lightweight game server networking library for Go. It provides a core network architecture and a series of tools an
Scalable Distributed Game Server Engine with Hot Swapping in Golang
GoWorld Scalable Distributed Game Server Engine with Hot Reload in Golang Features Architecture Introduction Get GoWorld Manage GoWorld Servers Demos
Skynet is a framework for distributed services in Go.
##Introduction Skynet is a communication protocol for building massively distributed apps in Go. It is not constrained to Go, so it will lend itself n
*DEPRECATED* Please use https://gopkg.in/redsync.v1 (https://github.com/go-redsync/redsync)
Redsync.go This package is being replaced with https://gopkg.in/redsync.v1. I will continue to maintain this package for a while so that its users do
The pure golang implementation of nanomsg (version 1, frozen)
mangos NOTE: This is the legacy version of mangos (v1). Users are encouraged to use mangos v2 instead if possible. No further development is taking pl
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.
What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are
A distributed lock service in Go using etcd
locker A distributed lock service client for etcd. What? Why? A distributed lock service is somewhat self-explanatory. Locking (mutexes) as a service
⟁ Tendermint Core (BFT Consensus) in Go
Tendermint Byzantine-Fault Tolerant State Machines. Or Blockchain, for short. Branch Tests Coverage Linting master Tendermint Core is Byzantine Fault
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services
sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w
Simplified distributed locking implementation using Redis
redislock Simplified distributed locking implementation using Redis. For more information, please see examples. Examples import ( "fmt" "time"
A library that implements the outboxer pattern in go
Outboxer Outboxer is a go library that implements the outbox pattern. Getting Started Outboxer was designed to simplify the tough work of orchestratin