Ready to deploy, distributed cryptocurrency trading bot

HyperTrade

Ready to deploy, distributed cryptocurrency trading bot.

Buy Me A Coffee Twitter: karan_6864 DigitalOcean Referral Badge

USE THIS SOFTWARE AT YOUR OWN RISK. THE AUTHOR ASSUMES NO LIABILITY FOR YOUR TRADING OUTCOMES.

charts dataframe portfolio config config-strategy charts-indicators telegram

๐Ÿ“– Contents

๐Ÿ” Overview

Here's a short overview of the project.

overview

๐Ÿ’ก Motivation

A few months ago I got into the crypto market. It was all new and fascinating to me. The idea of this project originally came from a script I used to automate buying and selling of fiat assets.

I was curious and wanted to scale it into a real system that can execute trades for me. I had a lot of fun building this. I got to play with lots of different technologies while growing my financial knowledge.

Feel free to reach out to me if you have any additional questions. There are lots of fixes and features to be done!

It is important to note that this project is under active development and was developed as an experiment. Currently, only Binance is supported but contributions are welcome!

Please leave a โญ as motivation or support by donating if you liked the idea ๐Ÿ˜„

โœจ Features

This system has lots of features such as:

โšก๏ธ Technologies

stack

โ“ Working

Here's a simple diagram illustrating how streams and events are published/subscribed in the system. We use NATS for messaging and streams, more about that in the architecture section.

Basically, The Exchange service publishes Kline/Candlestick data based on our configuration. Strategy service then subscribes to it and publishes a stream of dataframe which contains the indicators, buy/sell signals, and other metadata. The buy/sell signals are determined using the configuration as well, where we can customize our strategies given that they are supported.

Now, The Exchange service subscribes to the dataframe stream and executes trades on the signal using the Binance API. Any event such as order, trade, error, etc is published and then the Notification service can send a message to the user using the Telegram API.

Note: This only represents core events, there are more events for the Web and DB services.

working

๐Ÿญ Architecture

It's a microservices based architecture with event driven approach for decoupling. It uses distributed streams and messaging which keeps the system simple yet robust to make sure it can grow in the future.

architecture k8s

Note: Kubernetes diagram was generated using Lens - The Kubernetes IDE

Why so many technologies?

This started as an all Go project, but then I decided to add Python to the mix as it has a really good ecosystem for technical indicators and mathematical use cases.

For the web, I used React as it's easy and TypeScript provides some sanity to JavaScript projects!

Nginx and Postgres fit right in as the use case grew. As always, use what's right for the project!

Why event driven?

Originally I made this using HTTP REST, but systems like this are event driven by nature. Plus this approach helps to decouple services even more.

How do services communicate?

All the messaging infrastructure use cases are handled by NATS. Inter-service communications are mainly via publish-subscribe and request-reply patterns. Essentially, NATS acts as our service mesh!

We also use JetStream for real-time, persisted data streams.

publish-subscribe request-reply

Read more about Distributed communication patterns with NATS

Why Kubernetes? Isn't it overkill?

I agree! Kubernetes can be bit overkill, especially for this project. But my goal here was to keep it cloud agnostic, even though it was very tempting to just spin up AWS lambdas with event bridge.

๐Ÿ• Getting Started

Here we will setup our development environment. But first, we will need to install the following tools.

Tools

Steps

  • Once all the tools are installed, execute make prepare to prepare the local environment.
  • Create a infrastructure/k8s/env.yaml to similar structure as infrastructure/k8s/env.example.yaml and follow this guide.
  • Review the configuration in services/exchange/config.json
  • Start development with make dev command.
  • Use make stop to stop the local minikube cluster.

๐Ÿ›  Customization

We can customize trading strategies either via web app or services/exchange/config.json directly.

Currently, only few trading strategies like RSI and MACD are supported and I plan to add more soon.

config-strategy

๐Ÿ“š Usage

๐Ÿ’ฌ Telegram

Telegram bot helps us to interact with the system easily and receive real-time notifications.

Commands

The telegram bot supports the following commands:

telegram-commands
  • /configs: Get asset configurations.
  • /balance: Get current account balance.
  • /positions: Get actively held positions.
  • /stats: Get portfolio statistics.
  • /enable: Enable trading for a symbol.
  • /disable: Disable trading for a symbol.
  • /dump: Dump all the positions for a symbol.

Note: enable, disable and dump commands are symbol specific, and are executed as /cmd symbol. Example, /enable ETHUSDT

๐Ÿ’ป Web

Since this application deals with sensitive financial data, it is not recommended to expose it via ingress unless we have proper RBAC authorization in place. Hence, it is recommended to connect to it via port-forwarding on your local machine.

Steps

  • Connect to the application via port-forwarding using make connect command.

Note: Make sure doctl is authenticated, and we're using the correct k8s cluster name.

  • Open localhost:8080 to see the web interface.

  • Once done, use the make disconnect command to remove the kubectl context from your machine.

Note: It is recommended to use the telegram bot over the user interface on non-secure systems.

๐Ÿš€ Deployment

Deployments can be done via deploy.yml Github actions to our Digital Ocean Kubernetes cluster. But first, we will need to provision our infrastructure.

Important: Infrastructure we're about to provision has its own cost!

Tools

Steps

  • Create and export a GITHUB_TOKEN to access our Github container registry.
$ export TF_VAR_GHCR_TOKEN=value-of-your-token
  • Get an API token from DigitalOcean dashboard and export it temporarily for terraform.
$ export DIGITALOCEAN_TOKEN=value-of-your-token
  • Apply the Terraform configuration.
$ cd infrastructure
$ terraform init
$ terraform apply
  • Go to Github and add GHCR_TOKEN and DIGITALOCEAN_TOKEN to your repository secrets for Github actions.
  • Generate base64 string of your secrets, and add it as APP_SECRETS to your repository secrets for Github actions.
$ cat infrastructure/k8s/env.yaml | base64
  • Go to the actions tab and run the Build and Deploy action.

  • Once the deploy is complete, our application will be deployed in the hypertrade namespace on Kubernetes.

Note: If you want to change the name of the project, make sure to update all the associated Kubernetes manifest files, skaffold config, nginx config etc.

๐Ÿ‘ Contribute

Contributions are welcome as always, before submitting a new PR please make sure to open a new issue so community members can discuss it.

Additionally, you might find existing open issues which can help with improvements.

This project follows a standard code of conduct so that you can understand what actions will and will not be tolerated.

๐Ÿ“„ License

This project is GPL-3.0 licensed, as found in the LICENSE

Owner
Karan Pratap Singh
Software Engineer & Solutions Architect
Karan Pratap Singh
Comments
  • feat: migrate to ghcr

    feat: migrate to ghcr

    Changelog

    • Migrate from DO registry to GH registry to save operation cost
    • No more default namespace
    • Single node cluster (not the best, but It'll do)
    • Operational cost reduced to $10
  • Minimize Digital Ocean costs

    Minimize Digital Ocean costs

    Description

    Right now Digital Ocean can cost up to $30 / month due to the strange pricing model for containers registry.

    Implementation Plan

    Inline NATS service config to prevent storing containers? Thus making 1 less container to push, reducing cost by $15 (Again, strage pricing model!)

  • Run Locally in Minikube with Tilt

    Run Locally in Minikube with Tilt

    I really mess with this project. Its dope. I want to build on top of it, but I have trouble on a M1 Mac as Python is weird there.

    I've been using Tilt to have Python locally and live-reload (very much overkill but I think this project is fit)

    I think this would and could be a good use case for Tilt here.

  • Improve testing

    Improve testing

    Description

    Who writes tests for personal projects am I right? But seriously, we need to write some more tests! Especially integration tests, which can be challenging in event-driven systems.

  • Configurable order type

    Configurable order type

    Description

    Currently, every trade is executed as a market order. This can result in big losses due to the volatility of the crypto market ๐Ÿ˜ญ. So we need to support stop-loss orders to minimize risk and loss.

    Implementation Plan

    Sync and determine order state from Binance?

  • Switch base assets on demand

    Switch base assets on demand

    Description

    Currently, base assets are configured via config.json but they should also be configurable via the dashboard.

    Note: Currently only USDT works

    Implementation Plan

    • A simple dropdown can be used for base assets like USDT, BNB etc.?
    • Check if the base asset is available in balance first
  • Codegenerate events

    Codegenerate events

    Description

    Currently, events and data types have to be across services. They should be code generated from some sort of IDL

    Implementation Plan

    OpenAPI is a good fit here? But the use case is more close to how protocol buffers generate code (minus the protobuf stuff)

Cloud Native Electronic Trading System built on Kubernetes and Knative Eventing

Ingenium -- Still heavily in prototyping stage -- Ingenium is a cloud native electronic trading system built on top of Kubernetes and Knative Eventing

Aug 29, 2022
Fast, Docker-ready image processing server written in Go and libvips, with Thumbor URL syntax

Imagor Imagor is a fast, Docker-ready image processing server written in Go. Imagor uses one of the most efficient image processing library libvips (w

Dec 30, 2022
A minimal Go project with user authentication ready out of the box. All frontend assets should be less than 100 kB on every page load

Golang Base Project A minimal Golang project with user authentication ready out of the box. All frontend assets should be less than 100 kB on every pa

Jan 1, 2023
K8s-ingress-health-bot - A K8s Ingress Health Bot is a lightweight application to check the health of the ingress endpoints for a given kubernetes namespace.

k8s-ingress-health-bot A K8s Ingress Health Bot is a lightweight application to check the health of qualified ingress endpoints for a given kubernetes

Jan 2, 2022
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

Jan 5, 2023
Build and deploy Go applications on Kubernetes
Build and deploy Go applications on Kubernetes

ko: Easy Go Containers ko is a simple, fast container image builder for Go applications. It's ideal for use cases where your image contains a single G

Jan 5, 2023
Automatically deploy from GitHub to Replit, lightning fast โšก๏ธ

repl.deploy Automatically deploy from GitHub to Replit, lightning fast โšก๏ธ repl.deploy is split into A GitHub app, which listens for code changes and s

Dec 22, 2022
A tool to build, deploy, and release any environment using System Containers.
A tool to build, deploy, and release any environment using System Containers.

Bravetools Bravetools is an end-to-end System Container management utility. Bravetools makes it easy to configure, build, and deploy reproducible envi

Dec 14, 2022
A tool to build, deploy, and release any application on any platform.
A tool to build, deploy, and release any application on any platform.

Waypoint Website: https://www.waypointproject.io Tutorials: HashiCorp Learn Forum: Discuss Waypoint allows developers to define their application buil

Dec 28, 2022
Triggers an update to a Koyeb app service to re-deploy the latest docker image

Triggers an update to a Koyeb app service to re-deploy the latest docker image

May 5, 2021
Small and easy server for web-hooks to deploy software on push from gitlab/github/hg and so on

Deployment mini-service This mini web-server is made to deploy your code without yaml-files headache. If you just need to update your code somewhere a

Dec 4, 2022
Easily deploy your Go applications with Dokku.

dokku-go-example Easily deploy your Go applications with Dokku. Features: Deploy on your own server Auto deployment HTTPS Check the full step by step

Aug 21, 2022
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.

Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.

Jan 8, 2023
cloud native application deploy flow
cloud native application deploy flow

Triton-io/Triton English | ็ฎ€ไฝ“ไธญๆ–‡ Introduction Triton provides a cloud-native DeployFlow, which is safe, controllable, and policy-rich. For more introdu

May 28, 2022
A Go based deployment tool that allows the users to deploy the web application on the server using SSH information and pem file.

A Go based deployment tool that allows the users to deploy the web application on the server using SSH information and pem file. This application is intend for non tecnhincal users they can just open the GUI and given the server details just deploy.

Oct 16, 2021
Koyeb is a developer-friendly serverless platform to deploy apps globally.
Koyeb is a developer-friendly serverless platform to deploy apps globally.

Koyeb Serverless Platform Deploy a Go Gin application on Koyeb Learn more about Koyeb ยท Explore the documentation ยท Discover our tutorials About Koyeb

Nov 14, 2022
Deploy https certificates non-interactively to CDN services

certdeploy Deploy https certificates non-interactively to CDN services. Environment Variables CERT_PATH - Certificate file path, should contain certif

Nov 27, 2022
A simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app to Docker Hub

go-pipeline-demo A repository containing a simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app t

Nov 17, 2021
httpserver deploy in kubernetes

httpserver deploy in kubernetes cluster What is this? The project realizes the functions of mainstream httpserver based on golang / gin, including ele

Mar 15, 2022