Filter By
37 Resources
Golang Data Processing
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built
Dud is a lightweight tool for versioning data alongside source code and building data pipelines.
Dud Website | Install | Getting Started | Source Code Dud is a lightweight tool for versioning data alongside source code and building data pipelines.
CUE is an open source data constraint language which aims to simplify tasks involving defining and using data.
CUE is an open source data constraint language which aims to simplify tasks involving defining and using data.
Data syncing in golang for ClickHouse.
ClickHouse Data Synchromesh Data syncing in golang for ClickHouse. based on go-zero ARCH A typical data warehouse architecture design of data sync Aut
sq is a command line tool that provides jq-style access to structured data sources such as SQL databases, or document formats like CSV or Excel.
sq: swiss-army knife for data sq is a command line tool that provides jq-style access to structured data sources such as SQL databases, or document fo
A clean, safe, user-friendly implementation of GraphQL's Dataloader, written with generics in go
go-dataloader A clean, safe, user-friendly implementation of GraphQL's Dataloader, written with generics in go (go1.18beta1). Features written in gene
Graphik is a Backend as a Service implemented as an identity-aware document & graph database with support for gRPC and graphQL
Graphik is a Backend as a Service implemented as an identity-aware, permissioned, persistant document/graph database & pubsub server written in Go.
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data throughout the software development life cycle (SDLC) for engineering teams.
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
glow Purpose Glow is providing a library to easily compute in parallel threads or distributed to clusters of machines. This is written in pure Go. I a
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more
Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.
DataKit is collection agent for DataFlux.
DataKit DataKit is collection agent for DataFlux Build Dependencies apt-get install gcc-multilib: for building oracle input apt-get install tree: for
Declarative streaming ETL for mundane tasks, written in Go
Benthos is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads.
A stream processing API for Go (alpha)
A data stream processing API for Go (alpha) Automi is an API for processing streams of data using idiomatic Go. Using Automi, programs can process str
a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
monstache a go daemon that syncs mongodb to elasticsearch in realtime Version 6 This version of monstache is designed for MongoDB 3.6+ and Elasticsear
Machine is a library for creating data workflows.
Machine is a library for creating data workflows. These workflows can be either very concise or quite complex, even allowing for cycles for flows that need retry or self healing mechanisms.
A distributed, fault-tolerant pipeline for observability data
Table of Contents What Is Veneur? Use Case See Also Status Features Vendor And Backend Agnostic Modern Metrics Format (Or Others!) Global Aggregation
Open source framework for processing, monitoring, and alerting on time series data
Kapacitor Open source framework for processing, monitoring, and alerting on time series data Installation Kapacitor has two binaries: kapacitor – a CL
Kanzi is a modern, modular, expendable and efficient lossless data compressor implemented in Go.
kanzi Kanzi is a modern, modular, expendable and efficient lossless data compressor implemented in Go. modern: state-of-the-art algorithms are impleme
A babashka pod for interacting with sqlite3.
A babashka pod for interacting with sqlite3. Implemented using the Go go-sqlite3 and transit libraries.
Stream data into Google BigQuery concurrently using InsertAll() or BQ Storage.
bqwriter A Go package to write data into Google BigQuery concurrently with a high throughput. By default the InsertAll() API is used (REST API under t
Baker is a high performance, composable and extendable data-processing pipeline for the big data era
Baker is a high performance, composable and extendable data-processing pipeline for the big data era. It shines at converting, processing, extracting or storing records (structured data), applying whatever transformation between input and output through easy-to-write filters.
Convert struct, slice, array, map or others for Golang
XConv zh-CN XConv is a golang type convertor. It convert any value between types (base type, struct, array, slice, map, etc.) Features Convert between
xyr is a very lightweight, simple and powerful data ETL platform that helps you to query available data sources using SQL.
xyr [WIP] xyr is a very lightweight, simple and powerful data ETL platform that helps you to query available data sources using SQL. Supported Drivers
DEPRECATED: Data collection and processing made easy.
This project is deprecated. Please see this email for more details. Heka Data Acquisition and Processing Made Easy Heka is a tool for collecting and c
Bindings to rrdtool
Go bindings to rrdtool C library (rrdtool) This package implements Go (golang) bindings for the rrdtool C API. Installing rrd currently supports rrdto
Go Collection Stream API, inspired in Java 8 Stream.
GoStream gostream 是一个数据流式处理库。它可以声明式地对数据进行转换、过滤、排序、分组、收集，而无需关心操作细节。 Changelog 2021-11-18 add ToSet() collector Roadmap 移除go-linq依赖 Get GoStream go get
Map, Reduce, Filter, De/Multiplex, etc. for the Go language.
proto proto gives Go operations like Map, Reduce, Filter, De/Multiplex, etc. without sacrificing idiomatic harmony or speed. It also introduces a conv
Feed pipe input into a Discord server via webhook.
Feed pipe input into a Discord server via webhook.
Recommendation engine for Go
regommend Recommendation engine for Go Installation Make sure you have a working Go environment (Go 1.2 or higher is required). See the install instru
Several functional programming supporting in golang
A golang library that makes operations on slice easilier What can I do? slice process Map Filter Sort Reverse map process Keys Values output (starting
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.
Prometheus Common Data Exporter Prometheus Common Data Exporter 用于将多种来源(如http响应报文、本地文件、TCP响应报文、UDP响应报文)的Json、xml、yaml或其它格式的数据，解析为Prometheus metric数据。
churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline applications.
Churro - ETL for Kubernetes churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline appli
Simple CRUD application using CockroachDB and Go
Simple CRUD application using CockroachDB and Go
A library for performing data pipeline / ETL tasks in Go.
Ratchet A library for performing data pipeline / ETL tasks in Go. The Go programming language's simplicity, execution speed, and concurrency support m
This project is meant to make you code a digital version of an ant farm
This project is meant to make you code a digital version of an ant farm. Create a program lem-in that will read from a file (describing the ants and the colony) given in the arguments. Upon successfully finding the quickest path, lem-in will display the content of the file passed as argument and each move the ants make from room to room. How does it work? You make an ant farm with tunnels and rooms. You place the ants on one side and look at how they find the exit.
indodate is a plugin for golang programming language for date convertion on indonesian format
indodate is a package for golang programming language for date conversion on indonesian format