A single binary, simple, message queue.

MiniQueue

Tests

A stupid simple, single binary message queue using HTTP/2.

Most messaging workloads don't require enormous amounts of data, endless features or infinite scaling. Instead, they'd probably be better off with something dead simple.

MiniQueue is just that. A simple queue. You can publish bytes to topics and your consumers will receive what you published, nothing more.

Features

  • Simple to run
  • Very fast
  • Not infinitely scalable
  • Multiple topics
  • HTTP/2
  • Publish
  • Subscribe
  • Acknowledgements
  • Persistent
  • Prometheus metrics

API

  • POST /publish/:topic
  • GET /subscribe/:topic

Usage

Run MiniQueue where you would like.

It will then expose an HTTP/2 server used for publishing and consuming.

Benchmarks

As MiniQueue is under heavy development, take these benchmarks with a grain of salt. However, for those curious:

λ ~/ go-wrk -c 12 -d 10 -M POST -body "helloworld" https://localhost:8080/publish/test
Running 10s test @ https://localhost:8080/publish/test
  12 goroutine(s) running concurrently
104084 requests in 9.942585489s, 5.76MB read
Requests/sec:           10468.50
Transfer/sec:           592.94KB
Avg Req Time:           1.146295ms
Fastest Request:        262.281µs
Slowest Request:        867.958415ms
Number of Errors:       0

Running on my MacBook Pro (15-inch, 2019), with a 2.6 GHz 6-Core Intel Core i7.

Contributing

Contributors are more than welcome. Please feel free to open a PR to improve anything you don't like, or would like to add. No PR is too small!

License

This project is licensed under the MIT license.

Owner
Tom Arrell
Senior Backend Engineer @sumup — Lover of Rust and Go, hack around building keyboards when I'm bored
Tom Arrell
https://github.com/tomarrell/miniqueue
Similar Resources

RapidMQ is a pure, extremely productive, lightweight and reliable library for managing of the local messages queue

RapidMQ RapidMQ is a pure, extremely productive, lightweight and reliable library for managing of the local messages queue in the Go programming langu

Sep 27, 2022

redisqueue provides a producer and consumer of a queue that uses Redis streams

redisqueue redisqueue provides a producer and consumer of a queue that uses Redis streams. Features A Producer struct to make enqueuing messages easy.

Dec 29, 2022

dque is a fast, embedded, durable queue for Go

dque - a fast embedded durable queue for Go dque is: persistent -- survives program restarts scalable -- not limited by your RAM, but by your disk spa

Jan 8, 2023

Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Jan 4, 2023

Queue with NATS Jetstream to remove all the erlangs from cloud

Saf in Persian means Queue. One of the problems, that we face on projects with queues is deploying RabbitMQ on the cloud which brings us many challenges for CPU load, etc. I want to see how NATS with Jetstream can work as the queue to replace RabbitMQ.

Dec 15, 2022

A fast durable queue for Go

pqueue - a fast durable queue for Go pqueue is thread-safety, serves environments where more durability is required (e.g., outages last longer than me

Oct 16, 2022

Redis as backend for Queue Package

Redis as backend for Queue Package

redis Redis as backend for Queue package Setup start the redis server redis-server start the redis cluster, see the config # server 01 mkdir server01

Oct 16, 2022

NSQ as backend for Queue Package

NSQ as backend for Queue Package

NSQ as backend for Queue Package

Jul 4, 2022

Kudruk helps you to create queue channels and manage them gracefully.

kudruk Channels are widely used as queues. kudruk (means queue in Turkish) helps you to easily create queue with channel and manage the data in the qu

Feb 21, 2022
Comments
  • Subscription command issue?

    Subscription command issue?

    I love this queue -- go, simple, secure. Thank you. However I am having a terrible time getting subscriptions to work.

    curl --insecure --cacert certs/ca.key --key certs/node.key --cert certs/node.crt -X POST https://localhost:9999/subscribe/foo                                                     
{"error":"error decoding command"}

    Is it not possible to test subscriptions with curl? I publish just fine using the example in README. Thanks.

  • TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC

    TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC

    Hi tomarrell:

    I just clone the repository, and run the following:

    1. build the exetutable in the root directory
    go build .
    1. start the server
    ./miniqueue.exe --human
    1. publish a message to a topic using curl
    curl -X POST https://localhost:8080/publish/foo --data "helloworld"

    the server produce:

    2021/02/19 19:28:17 http: TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC

    and curl produce:

    curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

    the echo example has similar problem.

    maybe I should create a certificate on my own, and try again?

    I'm using Windows10 and git-bash

  • Allow subscription to topic which doesn't exist yet

    Allow subscription to topic which doesn't exist yet

    Currently if a subscriber attempts to read from a non-existent queue they will receive an error.

    This behaviour may be desirable in certain circumstances.

Related tags
Messaging miniqueue
A Multi Consumer per Message Queue with persistence and Queue Stages.
 A Multi Consumer per Message Queue with persistence and Queue Stages.

CrimsonQ A Multi Consumer per Message Queue with persistence and Queue Stages. Under Active Development Crimson Queue allows you to have multiple cons

Jul 30, 2022
Simple docker container to publish a fixed message to a specified queue. Created to be used with k8s CRON scheduling.

RabbitMQ Publish CRON Simple docker container to publish a fixed message to a specified rabbitmq exchange. Created to be used as part of a Kubernetes

Dec 20, 2021
Alertmanager go message broker - A simple message broker made to integrate with alertmanager/prometheus

Alertmanager message broker Prerequisites Go 1.16+ Sqllite driver About: The alertmanager message broker is a project made to meet some of my needs to

Dec 27, 2021
A lightweight, distributed and reliable message queue based on Redis

nmq A lightweight, distributed and reliable message queue based on Redis Get Started Download go get github.com/inuggets/nmq Usage import "github.com

Nov 22, 2021
KubeMQ is a Kubernetes native message queue broker

KubeMQ Community is the open-source version of KubeMQ, the Kubernetes native message broker. More about KubeMQ

Nov 20, 2021
Kafka implemented in Golang with built-in coordination (No ZooKeeper, single binary install, Cloud Native)

Jocko Distributed commit log service in Go that is wire compatible with Kafka. Created by @travisjeffery, continued by nash. Goals: Protocol compatibl

Aug 9, 2021
Asynq: simple, reliable, and efficient distributed task queue in Go
 Asynq: simple, reliable, and efficient distributed task queue in Go

Asynq Overview Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be sc

Dec 30, 2022
A simple persistent directory-backed FIFO queue.

pqueue pqueue is a simple persistent directory-backed FIFO queue. It provides the typical queue interface Enqueue and Dequeue and may store any byte s

Dec 12, 2022
Go simple async message bus
 Go simple async message bus

?? message-bus Go simple async message bus. ?? ABOUT Contributors: Rafał Lorenz Want to contribute ? Feel free to send pull requests! Have problems, b

Dec 29, 2022
Messagebus - Simple Message Bus Written in Golang

MessageBus Simple Message Bus Written in Golang How to Use go get gopkg.io/Usada

Apr 21, 2022