94 Resources
Golang parallel-computing Libraries
The fastest file download client
Pget - The fastest file download client Description Multi-Connection Download using parallel requests. Fast Resumable Cross-compiled (windows, linux,
ICPP 2022 (pap351) Under Review
SciCoFK: A Decentralized Scientific Computing Framework Embraced with Consensus and Incentive Mechanism This repo is the primary code of SciCoFK. In t
๐ parallel black box PostgreSQL unit tests run against a real database.
๐ psql-docker-tests-example Parallel black box PostgreSQL unit tests run against a real database. Consider reading the Medium Story first. This packa
Provides the function Parallel to create a synchronous in memory pipe and lets you write to and read from the pipe parallelly
iopipe provides the function Parallel to create a synchronous in memory pipe and lets you write to and read from the pipe parallely
M3u8-parallel-downloader - M3u8 parallel downloader with golang
m3u8-parallel-downloader Usage ./m3u8-parallel-downloader -input http://example.
Ordered-concurrently a library for parallel processing with ordered output in Go
Ordered-concurrently a library for parallel processing with ordered output in Go. Process work concurrently / in parallel and returns output in a channel in the order of input. It is useful in concurrently / parallelly processing items in a queue, and get output in the order provided by the queue.
Make functions return a channel for parallel processing via go routines.
This library helps you to transform any function into a function that returns a any type to a function that return such types within a channel. This is useful to run in parallel multiple functions and have control on the returned values.
A tool/library to run custom validations on Kubernetes resources in parallel
cluster-validator cluster-validator is a tool/library for performing resource validations in parallel on a Kubernetes cluster. For example, validating
Colonies is a generic framework for implementing next-generation distributed applications and systems
Colonies is a generic framework for implementing next-generation distributed applications and systems. It can be used as a building block for grid computing or edge computing, e.g. implement a meta operating system or cloud-of-cloud platform that combines many execution environments into a new virtual computing environment that can be controlled using an single unified API.
Command pattern for Go with thread safe serial and parallel dispatcher
Command Command pattern for Go with thread safe serial and parallel dispatcher. Installation go get -u github.com/txgruppi/command Tests go get -u -t
A simple package for executing work in parallel up to a limit.
concurrencylimiter A simple package for executing work concurrently - up to a limit. The intended usecase looks something like: func concurrentlyDo(ta
Industrial IoT Messaging and Device Management Platform
Mainflux Mainflux is modern, scalable, secure, open-source, and patent-free IoT cloud platform written in Go. It accepts user and thing (sensor, actua
Launch parallel processes (shuttles) with conditional parameter(s) specified in text file(s)
~# shuttle Launch some shuttles here and there. Usage Usage example: # Launch as many sqlmaps as the lines in targets.txt # with 4 maximum istances at
Edge Orchestration project is to implement distributed computing between Docker Container enabled devices.
Edge Orchestration Introduction The main purpose of Edge Orchestration project is to implement distributed computing between Docker Container enabled
EdgelessDB is a MySQL-compatible database for confidential computing
EdgelessDB is a MySQL-compatible database for confidential computing. It runs entirely inside a secure enclave and comes with advanced features for collaboration, recovery, and access control.
Parallel and Pipelined HTTP GET Utility
htcat is a utility to perform parallel, pipelined execution of a single HTTP GET. htcat is intended for the purpose of incantations like: htcat
Parallel processing through go routines, copy and delete thousands of key within some minutes
redis-dumper CLI Parallel processing through go routines, copy and delete thousands of key within some minutes copy data by key pattern from one redis
Fast Concurrent / Parallel Sorting in Go
sorty sorty is a type-specific, fast, efficient, concurrent / parallel QuickSort implementation (with an enhanced InsertionSort as subroutine). It is
A collection of useful utility functions
util A group of generic useful utility functions Dependencies parallelizer github.com/shomali11/parallelizer Examples Concurrency package main import
Crane (FinOps Crane) is an opensource project which manages cloud resource on Kubernetes stack, it is inspired by FinOps concepts.
Crane (FinOps Crane) is an opensource project which manages cloud resource on Kubernetes stack, it is inspired by FinOps concepts. Goal of Crane is to provide an one-stop shop project to help Kubernetes users to save cloud resource usage with a rich set of functionalities.
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.
What is Argo Workflows? Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflow
Godot: run sequential and parallel animations with less code
Anima Anima is an addon for Godot that allows you to create sequential and parallel animations with less code compared to Tween. Introduction Creating
Self-hosted web app for encoding files to a target format using distributed computing.
What is Encodarr? Encodarr is a self-hosted web application that encodes video files to a target format using distributed computing to spread the work
Stack Up is a simple deployment tool that performs given set of commands on multiple hosts in parallel.
Stack Up is a simple deployment tool that performs given set of commands on multiple hosts in parallel. It reads Supfile, a YAML configuration file, which defines networks (groups of hosts), commands and targets.
Execute multiple shell commands like Docker-Compose
parx parx is a simple tool to run multiple commands in parallel while having the output structured like Docker Compose does that. This is useful when
An easy-to-use Map Reduce Go parallel-computing framework inspired by 2021 6.824 lab1. It supports multiple workers on a single machine right now.
MapReduce This is an easy-to-use Map Reduce Go framework inspired by 2021 6.824 lab1. Feature Multiple workers on single machine right now. Easy to pa
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload. Run tools like masscan, puredns, ffuf, httpx or a
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.
The open-source cloud asset inventory backed by SQL. CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables. C
Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
Karmada Karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration Karmada (Kubernetes Armada) is a Kubernetes management system that enables
Pengenalan Concurrency dan Parallel Programming
Golang Goroutine Sumber Tutorial: Udemy Slide Pengenalan Concurrency dan Parallel Programming Pengenalan Parallel Programming Saat ini kita hidup dima
Parallel implementation of Gzip for modern multi-core machines written in Go
gzip Parallel implementation of gzip for modern multi-core machines written in Go Usage: gzip [OPTION]... [FILE] Compress or uncompress FILE (by defau
OpenYurt - Extending your native Kubernetes to edge(project under CNCF)
openyurtio/openyurt English | ็ฎไฝไธญๆ What is NEW! Latest Release: September 26th, 2021. OpenYurt v0.5.0. Please check the CHANGELOG for details. First R
Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.
Dataflow Summary Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines. Each pipeline is specified as a Kube
Microshift is a research project that is exploring how OpenShift1 Kubernetes can be optimized for small form factor and edge computing.
Microshift is a research project that is exploring how OpenShift1 Kubernetes can be optimized for small form factor and edge computing.
How we can run unit tests in parallel mode with failpoint injection taking effect and without injection race
This is a simple demo to show how we can run unit tests in parallel mode with failpoint injection taking effect and without injection race. The basic
mygopherhose - Parallel importer for mysqldumps.
mygopherhose Parallel importer for mysqldumps. What mygopherhose uses a dump produced by mysqldump and imports it trying to parallelize INSERT stateme
A very simple ssh-agent that signs requests in parallel
ssh-agent A very simple ssh-agent that signs requests in parallel. Usage To install and run the agent simply run: $ go install github.com/Woellchen/ss
Create links for game saves
GSLU - Game Save Linking Utility This is a utility program that I originally made for myself to help create links for my game saves. The best use I se
DSV Parallel Processor takes input files and query specification via a spec file
DSV Parallel Processor Spec file DSV Parallel Processor takes input files and query specification via a spec file (conventionally named "spec.toml").
Pulumi - Modern Infrastructure as Code. Any cloud, any language ๐
Pulumi's Infrastructure as Code SDK is the easiest way to create and deploy cloud software that use containers, serverless functions, hosted services,
GoPlus - The Go+ language for engineering, STEM education, and data science
The Go+ language for engineering, STEM education, and data science Summary about Go+ What are mainly impressions about Go+? A static typed language. F
Enable dynamic and seamless Kubernetes multi-cluster topologies
Enable dynamic and seamless Kubernetes multi-cluster topologies Explore the docs ยป View Demo ยท Report Bug ยท Request Feature About the project Liqo is
Golang implementation of distributed mutex on Azure lease blobs
Distributed Mutex on Azure Lease Blobs This package implements distributed lock available for multiple processes. Possible use-cases include exclusive
A simple API for computing diffs of your documents over the time built on a scalable technology stack.
Diffme API WIP - this is an API to compute diffs between documents. It serves as a way to easily create audit logs for documents in your system, think
Go library for structured parallelism
Go library for structured concurrency Structured concurrency helps reasoning about the behaviour of parallel programs. parallel implements structured
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload. Run tools like masscan, puredns, ffuf, httpx or anything you need and get results quickly!
Secure Edge Networking Based On Kubernetes And KubeEdge.
What is FabEdge FabEdge is an open source edge networking solution based on kubernetes and kubeedge. It solves the problems including complex network
Set up tasks to be executed in parallel.
A simple Go library to set up tasks to be executed in parallel. package main import ( "context" "log" "github.com/bep/workers" ) func main() {
An actor framework for Go
gosiris is an actor framework for Golang. Features Manage a hierarchy of actors (each actor has its own: state, behavior, mailbox, child actors) Deplo
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.
GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right
a small form factor OpenShift/Kubernetes optimized for edge computing
Microshift Microshift is OpenShift1 Kubernetes in a small form factor and optimized for edge computing. Edge devices deployed out in the field pose ve
Managing your Kubernetes clusters (including public, private, edge, etc) as easily as visiting the Internet
Clusternet Managing Your Clusters (including public, private, hybrid, edge, etc) as easily as Visiting the Internet. Clusternet (Cluster Internet) is
Antenna RPC is an RPC protocol for distributed computing, it's based on QUIC and Colfer. its currently an WIP.
aRPC - Antenna Remote Procedure Call Antenna remote procedure call (aRPC) is an RPC protocol focused on distributed processing and HPC. aRPC is implem
Workflow Orchestrator
Adagio - A Workflow Orchestrator This project is currently in a constant state of flux. Don't expect it to work. Thank you o/ Adagio is a workflow exe
Bigmachine is a library for self-managing serverless computing in Go
Bigmachine Bigmachine is a toolkit for building self-managing serverless applications in Go. Bigmachine provides an API that lets a driver process for
Securely access remote devices and servers
Deviceplane is an open source device management tool for embedded systems and edge computing. It solves various infrastructure problems related to rem
parallel: a Go Parallel Processing Library
parallel: a Go Parallel Processing Library Concurrency is hard. This library doesn't aim to make it easy, but it will hopefully make it a little less
A fully self-contained Nmap like parallel port scanning module in pure Golang that supports SYN-ACK (Silent Scans)
gomap What is gomap? Gomap is a fully self-contained nmap like module for Golang. Unlike other projects which provide nmap C bindings or rely on other
A serverless cluster computing system for the Go programming language
Bigslice Bigslice is a serverless cluster data processing system for Go. Bigslice exposes composable API that lets the user express data processing ta
Fast Raft framework using the Redis protocol for Go
This project has been archived. Please check out Uhaha for a fitter, happier, more productive Raft framework. Finn is a fast and simple framework for
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more
Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.
High-Performance server for NATS, the cloud native messaging system.
NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo
๐ฆ Streaming-Serverless Framework for Low-latency Edge Computing applications, running atop QUIC protocol, engaging 5G technology.
YoMo YoMo is an open-source Streaming Serverless Framework for building Low-latency Edge Computing applications. Built atop QUIC Transport Protocol an
A serverless cluster computing system for the Go programming language
Bigslice Bigslice is a serverless cluster data processing system for Go. Bigslice exposes composable API that lets the user express data processing ta
Kubernetes Native Edge Computing Framework (project under CNCF)
KubeEdge KubeEdge is built upon Kubernetes and extends native containerized application orchestration and device management to hosts at the Edge. It c
A flexible configuration manager for Wireguard networks
Drago A flexible configuration manager for WireGuard networks Drago is a flexible configuration manager for WireGuard networks which is designed to ma
GoPlus - The Go+ language for data science
GoPlus - The Go+ language for data science NOTE: Go+ is still under heavy developement. Please don't use it in production environment. Summary about G
GoPlus - The Go+ language for data science
GoPlus - The Go+ language for data science NOTE: Go+ is still under heavy developement. Please don't use it in production environment. Summary about G
EGo lets you build, debug und run Go apps on Intel SGX - as simple as conventional Go programming!
EGo lets you build, debug und run Go apps on Intel SGX - as simple as conventional Go programming!
EGo lets you build, debug und run Go apps on Intel SGX - as simple as conventional Go programming!
EGo is a framework for building confidential apps in Go. Confidential apps run in always-encrypted and verifiable enclaves on Intel SGX-enabled ha
Web-based Cloud Gaming service for Retro Game
CloudRetro provides an open-source cloud gaming platform for retro games. It started as an experiment for testing cloud gaming performance with WebRTC and libretro, and now it aims to deliver the most modern and convenient gaming experience through the technology.
An edge-native container management system for edge computing
SuperEdge is an open source container management system for edge computing to manage compute resources and container applications in multiple edge regions. These resources and applications, in the current approach, are managed as one single Kubernetes cluster. A native Kubernetes cluster can be easily converted to a SuperEdge cluster.
A library for parallel programming in Go
pargo A library for parallel programming in Go Package pargo provides functions and data structures for expressing parallel algorithms. While Go is pr
Parallel S3 and local filesystem execution tool.
s5cmd Overview s5cmd is a very fast S3 and local filesystem execution tool. It comes with support for a multitude of operations including tab completi
Sparse matrix formats for linear algebra supporting scientific and machine learning applications
Sparse matrix formats Implementations of selected sparse matrix formats for linear algebra supporting scientific and machine learning applications. Co
Linear algebra, eigenvalues, FFT, Bessel, elliptic, orthogonal polys, geometry, NURBS, numerical quadrature, 3D transfinite interpolation, random numbers, Mersenne twister, probability distributions, optimisation, differential equations.
Gosl - Go scientific library Gosl is a set of tools for developing scientific simulations using the Go language. We mainly consider the development of
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more
Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.
Sparse matrix formats for linear algebra supporting scientific and machine learning applications
Sparse matrix formats Implementations of selected sparse matrix formats for linear algebra supporting scientific and machine learning applications. Co
Golang RServe client. Use R from Go
Roger Roger is a Go RServe client, allowing the capabilities of R to be used from Go applications. The communication between Go and R is via TCP. It i
Evolutionary optimization library for Go (genetic algorithm, partical swarm optimization, differential evolution)
eaopt is an evolutionary optimization library Table of Contents Changelog Example Background Features Usage General advice Genetic algorithms Overview
Industrial IoT Messaging and Device Management Platform
Mainflux Mainflux is modern, scalable, secure, open-source, and patent-free IoT cloud platform written in Go. It accepts user and thing (sensor, actua
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built
Run functions in parallel :comet:
Parallel fn Run functions in parallel. Limit the number of goroutines running at the same time. Installation go get -u github.com/rafaeljesus/parallel
Floc: Orchestrate goroutines with ease.
go-floc Floc: Orchestrate goroutines with ease. The goal of the project is to make the process of running goroutines in parallel and synchronizing the
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services
sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w
High-Performance server for NATS, the cloud native messaging system.
NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo
Parallel Digital Universe - A decentralized identity-based social network
Parallel Digital Universe Golang implementation of PDU. What is PDU? Usage Development Contributing PDU PDU is a decentralized identity-based social n
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built
Go parallel gzip (de)compression
pgzip Go parallel gzip compression/decompression. This is a fully gzip compatible drop in replacement for "compress/gzip". This will split compression
Next generation distributed, event-driven, parallel config management!
mgmt: next generation config management! About: Mgmt is a real-time automation tool. It is familiar to existing configuration management software, but
A parallel downloader with resume capability
Grozilla The Grozilla is a simple implementation that allows downloading of video,audio,package or zip files parallely and efficiently using light wei
Dependency-free replacement for GNU parallel, perfect fit for usage in an initramfs.
coshell v0.2.5 A no-frills dependency-free replacement for GNU parallel, perfect for initramfs usage. Licensed under GNU/GPL v2. How it works An sh -c
ops - build and run nanos unikernels
OPS Ops is a tool for creating and running a Nanos unikernel. It is used to package, create and run your application as a nanos unikernel instance. Ch