Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

Cadence

Build Status Coverage Status Slack Status

Visit cadenceworkflow.io to learn about Cadence.

This repo contains the source code of the Cadence server. To implement workflows, activities and worker use Go client or Java client.

See Maxim's talk at Data@Scale Conference for an architectural overview of Cadence.

Getting Started

Start the cadence-server locally

We highly recommend that you use Cadence service docker to run the service.

Run the Samples

Try out the sample recipes for Go or Java to get started.

Client SDKs

Java and Golang clients are developed by Cadence team:

Other clients are developed by community:

Use CLI Tools

Use Cadence command-line tool to perform various tasks on Cadence server cluster

For manual setup or upgrading server schema --

  • If server runs with Cassandra, Use Cadence Cassandra tool to perform various tasks on database schema of Cassandra persistence
  • If server runs with SQL database, Use Cadence SQL tool to perform various tasks on database schema of SQL based persistence

TIPS: Run make tools to build all tools mentioned above.

NOTE: See CONTRIBUTING for prerequisite of make command.

Use Cadence Web

Try out Cadence Web UI to view your workflows on Cadence. (This is already available at localhost:8088 if you run Cadence with docker compose)

Documentation

Visit cadenceworkflow.io for documentation.

Join us in Cadence Docs project. Raise an Issue or Pull Request there.

Getting Help

Contributing

We'd love your help in making Cadence great. Please review our contribution guide.

If you'd like to propose a new feature, first join the Slack channel to start a discussion and check if there are existing design discussions. Also peruse our design docs in case a feature has been designed but not yet implemented. Once you're sure the proposal is not covered elsewhere, please follow our proposal instructions.

License

MIT License, please see LICENSE for details.

Owner
Uber Open Source
Open Source Software at Uber
Uber Open Source
Comments
  • Fix make install-schema-es-v6  and install-schema-es-v7

    Fix make install-schema-es-v6 and install-schema-es-v7

    What changed? Fix make install-schema-es-v6 and install-schema-es-v7 command, it will template first then create index using the template. The only change is using static path to the template file.

    Why?

    How did you test it? Run the command locally

    Potential risks

    Release notes

    Documentation Changes

  • updating dependencies.

    updating dependencies.

    What changed?

    Updating cadence dependencies mostly newer versions.

    Why? Best practice to keep dependencies up to date in order to avoid security issues and to benefit from improvements made in the dependencies.

    How did you test it?

    Pipeline tests.

    Potential risks

    Build issues, code issues due to changed functionality of some dependencies.

    Release notes Updating a large part of dependencies to newer versions.

    Documentation Changes

  • Hot shard detection

    Hot shard detection

    Hot-shard detection

    Summary

    Samples tasks being processed and issues a log and metric if a repeated entry is identified, with the intention of spotting hot-shards.

    Background

    Based on a lot of operational experiences, it has been clear for a while that we need a cheap way to identify workflows which are running continue-as-new a lot or have very high task process thrashing on a single shard, such activities can take out a database's availbility. It's also operationally not always quite easy to identify such workflows without more subtle-clues such as looking at timeout correlations and guesswork.

    Initially I was considering a more sophisticated detector - something which would be able to identify if a workflow has ever been seen with some approximate count and considered count-min-sketch implementations for this. However, after fiddling with them for a while I decided against them in favour of a far simpler approach of just sampling. The reason for this is that count-min-sketch and similar probabilistic approximation algorithms overcount, making the risks of false-positives everpresent unless you have a moderate amount of memory allocated for such detection.

    Sampling on the other hand, is, if tuned correctly - by definition - undercounts, making it easier to control, and - I hope - offers a similarly modest memory footprint.

    Testing

    So far it's only tuned to my laptop, and given it's all configured with absolute values, this is near-certainly wrong. I'll have to tune more once deployed.

    Potential risks Lots, a panic in this could do quite a bit of damage, it'll have to be tested well.

    Release notes Ready for release, no changes required.

    Documentation Changes

  • updating some dependencies.

    updating some dependencies.

    What changed? Updating a few dependencies.

    Why?

    In general good practice to keep dependencies up to date to avoid security vulnerabilities in old versions and to get benefits of new features/improved performance.

    How did you test it?

    Relying on tests.

    Potential risks

    Some of the new dependency versions might now work correctly, work differently. There is always a risk when upgrading dependencies.

    Release notes

    Documentation Changes

  • Delete uninitialized workflow execution record if workflow failed to start

    Delete uninitialized workflow execution record if workflow failed to start

    What changed? Added new delete uninitialized workflow execution method If workflow failed to start, delete the record. We only need to keep the records for the stuck workflows

    Why?

    How did you test it? Tested locally

    Potential risks

    Release notes

    Documentation Changes

dque is a fast, embedded, durable queue for Go

dque - a fast embedded durable queue for Go dque is: persistent -- survives program restarts scalable -- not limited by your RAM, but by your disk spa

Jan 8, 2023
A fast durable queue for Go

pqueue - a fast durable queue for Go pqueue is thread-safety, serves environments where more durability is required (e.g., outages last longer than me

Oct 16, 2022
A simple queueing system for long-running commands

qme (queue me) A simple queueing system for long-running commands. It allows you to queue up shell commands from anywhere, and run them in order. This

Nov 18, 2022
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Jan 7, 2023
A wrapper of streadway/amqp that provides reconnection logic and sane defaults

go-rabbitmq Wrapper of streadway/amqp that provides reconnection logic and sane defaults. Hit the project with a star if you find it useful ⭐ Supporte

Dec 28, 2022
Scalable real-time messaging server in language-agnostic way
Scalable real-time messaging server in language-agnostic way

Centrifugo is a scalable real-time messaging server in language-agnostic way. Centrifugo works in conjunction with application backend written in any

Jan 2, 2023
GTA(Go Task Async) is a lightweight reliable asynchronous task and transaction message library for Golang

GTA (Go Task Async) is a lightweight and reliable asynchronous task and transaction message library for by golang.

Jun 4, 2022
💨 A real time messaging system to build a scalable in-app notifications, multiplayer games, chat apps in web and mobile apps.
💨 A real time messaging system to build a scalable in-app notifications, multiplayer games, chat apps in web and mobile apps.

Beaver A Real Time Messaging Server. Beaver is a real-time messaging server. With beaver you can easily build scalable in-app notifications, realtime

Jan 1, 2023
Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda
Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda

Reference Architecture using SingleStore and Redpanda for global logistics ?? INFO: For the story behind this code (and epic dashboards), check out th

Oct 29, 2022
Emits events in Go way, with wildcard, predicates, cancellation possibilities and many other good wins

Emitter The emitter package implements a channel-based pubsub pattern. The design goals are to use Golang concurrency model instead of flat callbacks

Jan 4, 2023
Declare AMQP entities like queues, producers, and consumers in a declarative way. Can be used to work with RabbitMQ.

About This package provides an ability to encapsulate creation and configuration of RabbitMQ([AMQP])(https://www.amqp.org) entities like queues, excha

Dec 28, 2022
Imagine a simple, beautiful, secure, and freaking fast way to broadcast your events throw the internet

Event Superintendent Imagine a simple, beautiful, secure, and freaking fast way to broadcast your events throw the internet. That's exactly why you sh

Jul 18, 2022
Asynq: simple, reliable, and efficient distributed task queue in Go
Asynq: simple, reliable, and efficient distributed task queue in Go

Asynq Overview Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be sc

Dec 30, 2022
Inspr is an application mesh for simple, fast and secure development of distributed applications.
Inspr is an application mesh for simple, fast and secure development of distributed applications.

Inspr is an engine for running distributed applications, using multiple communication patterns such as pub sub and more, focused on type consistency a

Jun 10, 2022
A lightweight, distributed and reliable message queue based on Redis

nmq A lightweight, distributed and reliable message queue based on Redis Get Started Download go get github.com/inuggets/nmq Usage import "github.com

Nov 22, 2021
A realtime distributed messaging platform
A realtime distributed messaging platform

Source: https://github.com/nsqio/nsq Issues: https://github.com/nsqio/nsq/issues Mailing List: [email protected] IRC: #nsq on freenode Docs:

Dec 29, 2022
Easy to use distributed event bus similar to Kafka
Easy to use distributed event bus similar to Kafka

chukcha Easy to use distributed event bus similar to Kafka. The event bus is designed to be used as a persistent intermediate storage buffer for any k

Dec 30, 2022
Distributed Lab 3: Message Broker in Go
Distributed Lab 3: Message Broker in Go

Distributed Lab 3: Message Broker in Go Using the lab sheet There are two ways to use the lab sheet, you can either: create a new repo from this templ

Oct 29, 2021
⚡️ A lightweight service that will build and store your go projects binaries, Integrated with Github, Gitlab, Bitbucket and Bitbucket Server.
⚡️ A lightweight service that will build and store your go projects binaries, Integrated with Github, Gitlab, Bitbucket and  Bitbucket Server.

Rabbit A lightweight service that will build and store your go projects binaries. Rabbit is a lightweight service that will build and store your go pr

Nov 19, 2022