Stream new blocks to various services (redis/elasticsearch/...)

AlgoNode algostreamer utility

About algostreamer

Small utility to stream past and/or current Algorand node JSON blocks to Redis or stdout.

About AlgoNode

We are here to help your Algorand node shine! Go to https://algonode.cloud for more.

Install

go get github.com/algonode/algostreamer

Config

config.json

{
  "algod" : {
    "address" : "http://localhost:8080",
    "token" : "...",
    "queue" : 100
  },
  "redis": {
    "addr": "localhost",
    "user": "",
    "pass": "",
    "db": 10
  }
}
  • You can find your token in node/data/algo.token
  • You can find your address in node/data/alogo.net

Run

Start streaming from the current block

./algostreamer -f config.json -s 2>>stream.log

Start streming from the block no 18000000 and then continue with current blocks

./algostreamer -r 18000000 -f config.json -s 2>>stream.log

WARN: Redis is not yet implemented

License

Copyright (C) 2022 AlgoNode Org.

algostreamer is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Support AlgoNode

if you like what we do feel free to support us by sending some microAlgos to

AlgoNode wallet: S322JRT4RZ4L2CLEQ5HXQBU2CNH3DLLO6JJEMWGPQHAOG2ALCH7ZAHXOPE

Owner
AlgoNode
One-stop shop for Algorand node & indexer running
AlgoNode
Similar Resources

Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)

gokv Simple key-value store abstraction and implementations for Go Contents Features Simple interface Implementations Value types Marshal formats Road

Dec 24, 2022

godis - an old Redis client for Go

godis Implements a few database clients for Redis. There is a stable client and an experimental client, redis and exp, respectively. To use any of the

Apr 16, 2022

Google Go Client and Connectors for Redis

Go-Redis Go Clients and Connectors for Redis. The initial release provides the interface and implementation supporting the (~) full set of current Red

Oct 25, 2022

Redis client library for Go

go-redis go-redis is a Redis client library for the Go programming language. It's built on the skeleton of gomemcache. It is safe to use by multiple g

Nov 8, 2022

Type-safe Redis client for Golang

Redis client for Golang ❤️ Uptrace.dev - distributed traces, logs, and errors in one place Join Discord to ask questions. Documentation Reference Exam

Jan 4, 2023

Redis Sorted Sets Benchmark

redis-zbench-go Redis Sorted Sets Benchmark Overview This repo contains code to trigger load ( ZADD ) or query (ZRANGEBYLEX key min max) benchmarks, w

May 18, 2021

Use Redis' MONITOR to draw things in a terminal

Use Redis' MONITOR to draw things in a terminal

Redis Top Redistop uses MONITOR to watch Redis commands and shows per command and per host statistics. Because MONITOR streams back all commands, its

Aug 30, 2022

Examples and code to assign a name to your MongoDB, MySQL, PostgreSQL, RabbitMQ, and redis connection.

Examples and code to assign a name to your MongoDB, MySQL, PostgreSQL, RabbitMQ, and redis connection.

your connection deserves a name 👀 When your app interacts with an external system, assign a name to the connection. An external system in this contex

Dec 14, 2022

redis protocol server for go.

redigosrv what redis server侧的golang版本实现。 why 作为后端RD,你开发的,维护的最多的应该就是你的server,作为服务的提供方来说,想要和客户端进行有效的交互,首先要要能理解信息的含义,因此一套通信协议是必须的。 为什么选择redis协议,应用层有很多成熟的

Nov 30, 2021
Comments
  • remove protocol from install command

    remove protocol from install command

    the current install command resulted in go get: malformed module path "https:/github.com/algonode/algostreamer": invalid char ':' on my linux machine, it may be a platform specific thing

    $ go get https://github.com/algonode/algostreamer
    go get: malformed module path "https:/github.com/algonode/algostreamer": invalid char ':'
    
  • Basic MQTT support

    Basic MQTT support

    Built on top of AlgoNode/algostreamer#2 Adds basic MQTT support, together with docker composeability.

    One thing to note here: I didn't spend much time on how to avoid spawning multiple streams (it's one per sink now) -- this could be sorted out with some sort of channel broadcasting.

  • Demo env with docker compose

    Demo env with docker compose

    It's always good to have a simple way to spin up the demo environment. Configured it together with Redis and RedisInsight. Docker compose makes it super convenient for everyone. README file updated. EditorConfig is optional but I just add it to everything I contribute to.

    upgrade Go to v1.19 and all the dependencies -- mainly due to outdated go-algorand causing the following error: [!ERR][REDIS] consensus protocol https://github.com/algorandfoundation/specs/tree/44fa607d6051730f5264526bf3c108d51f0eadb6 not found

Redis client Mock Provide mock test for redis query

Redis client Mock Provide mock test for redis query, Compatible with github.com/go-redis/redis/v8 Install Confirm that you are using redis.Client the

Dec 27, 2022
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.

GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right

Apr 27, 2022
Bxd redis benchmark - Redis benchmark tool for golang

使用 redis benchmark 工具, 测试 10 20 50 100 200 1k 5k 字节 value 大小,redis get set 性能。 r

Jan 22, 2022
High-performance framework for building redis-protocol compatible TCP servers/services

Redeo The high-performance Swiss Army Knife for building redis-protocol compatible servers/services. Parts This repository is organised into multiple

Jan 4, 2023
Stream data into Google BigQuery concurrently using InsertAll()

Kik and me (@oryband) are no longer maintaining this repository. Thanks for all the contributions. You are welcome to fork and continue development. B

Dec 7, 2022
Quick demo of Neo4j 4.4 new impersonation feature in Go

Neo4j 4.4 impersonation demo This is a simple application that demonstrates how to configure impersonation. Database setup Start a Neo4j server. On th

Dec 13, 2021
redis client implement by golang, inspired by jedis.

godis redis client implement by golang, refers to jedis. this library implements most of redis command, include normal redis command, cluster command,

Dec 6, 2022
Go client for Redis

Redigo Redigo is a Go client for the Redis database. Features A Print-like API with support for all Redis commands. Pipelining, including pipelined tr

Jan 1, 2023
Type-safe Redis client for Golang

Redis client for Golang ❤️ Uptrace.dev - distributed traces, logs, and errors in one place Join Discord to ask questions. Documentation Reference Exam

Jan 1, 2023
Go Redis Client

xredis Built on top of github.com/garyburd/redigo with the idea to simplify creating a Redis client, provide type safe calls and encapsulate the low l

Sep 26, 2022