Business Process eXecution Engine


Problem: creating and maintaining robust business systems

Creating a proof-of-concept business system is relatively easy. Happy execution path, no catastrophic failures, not much thought given to reliability. Data is easily kept in a few tables. Scale is not important.

This all starts changing, however, when this starts maturing. The logic needs to be maintained and kept in sync with business needs. Corner cases nobody thought of or tested for keep appearing. Failures do occur. Incosistent state happens. Suddenly (but arguably, expectably), developers are burdended with a lot more work. Onboarding becomes more difficult as understanding the intricacies of the system takes longer.

Ultimately, the cost of development and shipping keeps rising, while velocity is coming down. And that's a problem.


BPXE: Business Process eXecution Engine

BPXE is a BPMN 2.0-based business process execution engine and is a proposed solution to the problem statement above.

BPMN stands for Business Process Model and Notation. It is a specification that defines visual notation and execution semantics.

BPXE focuses on the execution aspect of such notation, effectively allowing the processes described in BPMN to function as if they were programs. BPXE is not the only such engine, as there are many commercially or community supported ones.

Having visualized processes that are also determinisitcally executable is key to maintaining a coherent understanding of what the process is supposed to do across teams and specialties.

Processes as Source of Truth

Equally important aspect of BPXE is that it makes processes and their executions a durable source of truth. This means that process instances can query previous executions of any processes to make further decisions.

As an example, consider a Purchasing process, which chooses a special path if it queries previous execution and finds out that when given the same or similar shopping cart, majority of those process executions were abandoned (i.e. customer did not complete a purchase). This kind of logic can be easily integrated into a process and updated as needed, giving a much better level of insight and control at a much lower modification cost.

Goals

  • Reasonably good performance
  • Small footprint
  • Multiplatform (servers, browsers, microcontrollers)
  • Multitenancy capabilities
  • Distributed process execution
  • Semantic correctness
  • Failure resistance

Usage

At this time, BPXE does not have an executable server of its own and can be only used as a Go library.

Licensing & Contributions

BPXE is Open Source software in the making. Its source code is currently available under the terms of Business Source License 1.1 with an Additional Use Grant for non-commercial purposes. Moreover, according to the terms of this license, every release of BPXE will eventually be re-licensed under the terms of Apache 2.0 license, on its fourth anniversary.

We also take contributions under the terms of New BSD license or in public domain.

Owner
BPXE
Business Process eXecution Engine
BPXE
Comments
  • Mock clock leaves lock taken

    Mock clock leaves lock taken

    This pattern repeats:

    https://github.com/bpxe/bpxe/blob/0828554a14ebbdc2c38b7564443c0036b0f05ab0/pkg/clock/mock.go#L53-L59

    	m.Lock()
    	ch := make(chan time.Time, 1)
    	if duration.Nanoseconds() <= 0 {
    		ch <- m.now
    		close(ch)
    		return ch
    	}
    

    This leaves the lock taken. Should use

    m.Lock()
    defer m.Unlock()
    
  • Outsiders cannot set issue labels

    Outsiders cannot set issue labels

    https://github.com/bpxe/bpxe/blob/master/CODEREVIEWS.md#how-do-i-submit-reviews

    Simply open an issue and assign the code review label to it.

    Without extra access to the repository, people cannot do that.

  • Impossible clock logic

    Impossible clock logic

    https://github.com/bpxe/bpxe/blob/0828554a14ebbdc2c38b7564443c0036b0f05ab0/pkg/clock/host_generic.go#L28

    				if t1.Before(t1) {
    

    You can be pretty darn sure that t1 is never before itself.

  • Add license scan report and status

    Add license scan report and status

    Your FOSSA integration was successful! Attached in this PR is a badge and license report to track scan status in your README.

    Below are docs for integrating FOSSA license checks into your CI:

  • Problem: passing instance information into traces

    Problem: passing instance information into traces

    Problem: passing instance information into traces

    Currently, traces produced by many components don't point to particular process instances they are produced in. This will make results of future analysis or querying of such information ambiguous.

    Solution: allow traces to be wrapped and unwrapped

    This change introduces the concept of trace wrapping (primarily through the newly introduced WrappedTrace interface and unwrapping (though WrappedTrace.Unwrap() and tracing.Unwrap() functions).

    Tracers can be contextualized by creating a new tracer that will transform produced traces (see tracing.NewTraceTransformingTracer) and passing it to components in a given context.

    Also, there's another problem that have been observed during the development of this feature:

    Problem: process instance observing traces from outside

    Currently, there's no reason for process instance components to observe traces from outside of its own scope; in fact, it'll likely mistreat traces from other instances or other processes as its own.

    Solution: compartmentalize tracers

    This way, instance gets its own tracer and its tracer are relayed to process's tracer and so on. This removes transforming tracer as it is no longer necessary. Instead, tracer relay does its job.

    I am still not 100% confident this is the right approach but it is better than leaving things as they are with scopes bleeding into other scopes. I am trying to think if there's any reason for observing traces outside of tracing compartment. If there will be such a need, we'll need to wire it through accordingly.

  • Finish adding @fire-bot to your repo

    Finish adding @fire-bot to your repo

    Greetings, fire here!

    @yrashk recently invited @fire-bot to this repository. Before fire is enabled, @yrashk needs to complete a few steps:

    Complete fire setup now

    Only @yrashk will be able to enable fire using the above link. If someone added fire by mistake, feel free to remove @fire-bot from your repo's collaborators.

  • Problem: tracking mutex unlocking patterns

    Problem: tracking mutex unlocking patterns

    Some of the code has branching with its own unlocking of mutexes. This code is error-prone as it's easy to forget to unlock.

    Solution: use deferred unlocks where feasible

    Previous implementation was based on a now-outdated knowledge that deferrals are slower. As it was pointed out in #68, this is no longer true as of Go 1.14. My own measurements seem to confirm this as well.

    So, where it was reasonable, I've rewritten unlocking to happen in deferred statements.

  • Problem: bad advice on testing clock changes

    Problem: bad advice on testing clock changes

    It suggests to change time within a docker container, but it'll actually change system time, because we give that capability to the container.

    Solution: rewrite the advice to use a virtual machine and provide a Vagrant setup for that, as well as instructions.

  • Problem: `clock.Mock.After` doesn't unlock

    Problem: `clock.Mock.After` doesn't unlock

    When After takes the fast path of returning when the duration is zero or negative, it doesn't unlock its mutex.

    Solution: unlock it (just like Until does)

    Addresses #68

  • Problem: impossible clock logic in host's change monitor

    Problem: impossible clock logic in host's change monitor

    There's a test that tests whether a certain time is before itself. That's clearly impossible.

    Solution: check whether it's before previously known time

    While we're at it, update current time on every iteration of the loop.

    The reason for this mishap is that this piece of code is not well tested yet (since it can be only tested manually)

    Addresses #64

  • Problem: timer tests fail sporadically

    Problem: timer tests fail sporadically

    The error seems to indicate that something is being received from the channel when it's not expected to.

    Solution: handle channel closure

    Since timers close channels when it is safe to do so, sometimes tests receive another (zero) message that indicates channel closure.

    This change handles this behavior by observing optional flag that indicates whether the channel is closed.

  • Proposal: split server and library, means provide flexibility in usage

    Proposal: split server and library, means provide flexibility in usage

    Hi,

    I really appreciate your work in building such a BPMN engine in Go. I'm quite impressed by your problem statement because it's exactly my thinking. Really great job done 👍

    One sentence caught my attention - the usage one: "BPXE does not have an executable server of its own and can be only used as a Go library." Frankly said that's exactly, what I'm looking for, an engine as a library.

    Let me explain, why I think splitting BPXE into lib and server as separate projects would be a better idea. Every point you mentioned about business is interested in transparency and wants to know what is implemented is right. So the benefits of BPMN having a modeler and thus some "live documentation" is great. On the other hand, when you want to deploy a (or any) workflow engine into a server, you're faced with questions about partitioning, consistency, availability (CAP theorem). What I mean: as the author of this project, you would need to support the most common database, fiddle with their quirks, ensure multiple server instances can run (high availability), and make this overall very responsive. That's a lot of work, which not everyone needs.

    Think for a moment, bpxe would have no DB persistency implemented and just allows other developers to load BPMN files, and execute them in memory only - period. This would lower the barrier to include such a lib instead of developing homebrew state machines or what-not, with having full documentation and modeling support available.

    That said, the bpxe engine would be focused on being a lib without DB support, one would certainly need two handfuls of callbacks/hooks so that application developers can react to state changes of the BPMN process state. They would follow their existing paradigms of putting the state into the DB with their preferred DB technology.

    Bare with me, when I share this example from the Javascript space ... https://github.com/paed01/bpmn-engine/blob/HEAD/docs/Examples.md This engine is built as lib only.

    Considering the lib offering such interfaces, you're not blocked and still be able to develop another project, which then would/could focus on being a server incl. database support, UI etc.

    I hope my thought is not offending, because I really love Open Source and collaboration around it. And I would be very glad to get some reflections from you and this idea.

    Sincerely Martin

  • Type aliases as used here don't provide value, type safety

    Type aliases as used here don't provide value, type safety

    https://github.com/bpxe/bpxe/blob/0828554a14ebbdc2c38b7564443c0036b0f05ab0/pkg/bpmn/schema.go#L19-L29

    type QName = string
    type Id = string
    

    This lets callers intermingle the types freely, removing any benefit of using the types. Should use

    type QName string
    type Id string
    

    and so on.

    https://play.golang.org/p/k8vgBkqdmwr

    package main
    
    import (
    	"fmt"
    )
    
    type Foo = string
    type Bar = string
    
    func demo(b Bar) {
    	fmt.Printf("i'm happy to have receiver a bar: %v\n", b)
    }
    
    func main() {
    	var f Foo = "i'm a foo"
    	demo(f)
    }
    

    Also, ID not Id: https://github.com/golang/go/wiki/CodeReviewComments#initialisms

  • Problem: testing different cases

    Problem: testing different cases

    Currently, testing BPXE against different cases / BPMN documents is a bit verbose, and it's sometimes hard to debug.

    Proposed solution (1): extract more primitives to simplify testing

    Proposed solution (2): externalize testing by creating a common tracing format as well as live querying / event injection.

    This is a larger undertaking than (1) but it has the following benefits:

    1. [Live] trace querying is one of the most important features BPXE will provide, so it needs to be done anyway
    2. It will allow to test any BPMN engine that can produce the same tracing format.

    I think proposed solutions (1) and (2) aren't exactly mutually exclusive.

  • Problem: can't instantiate process using Receive Task

    Problem: can't instantiate process using Receive Task

    As mentioned in 460473db4420e2c14185afe65dcba3f8a8e2a1a6, one can't instantiate a process using a Receive Task, simply because Receive Task hasn't been implemented.

  • Problem: can't instantiate process via event-based gateway

    Problem: can't instantiate process via event-based gateway

    As mentioned in 460473db4420e2c14185afe65dcba3f8a8e2a1a6, one can't instantiate a process using an Event-Based Gateway, simply because it's not a handled use case yet.

  • Problem: event-based gateway testing

    Problem: event-based gateway testing

    Currently (as of 0154b48a964c93f02947029be367730974cea4f1) https://github.com/bpxe/bpxe/blob/0154b48a964c93f02947029be367730974cea4f1/pkg/flow_node/gateway/event_based_gateway/tests/event_based_gateway_test.go#L18-L25 only tests event-based gateway that it'll follow a certain path in presence of just one event, but it doesn't test what will happen if both events will occur.

    Proposed solution: figure out how to improve the test to support this case

    Naive implementation like this one below blocks somewhere:

    testEventBasedGateway(t, func(reached map[string]int) {
        assert.True(t, reached["task1"] == 1 || reached["task2"] == 1)
        assert.Equal(t, 1, reached["end"])
    }, event.NewMessageEvent("Msg1", nil), event.NewSignalEvent("Sig1"))
    

    Not yet sure if this is due to how the test function works or due to a bug in the implementation.

Tool for shell commands execution, visualization and alerting. Configured with a simple YAML file.
Tool for shell commands execution, visualization and alerting. Configured with a simple YAML file.

Sampler. Visualization for any shell command. Sampler is a tool for shell commands execution, visualization and alerting. Configured with a simple YAM

Dec 28, 2022
A CLI command to parse Terraform execution result and notify it to Backlog
A CLI command to parse Terraform execution result and notify it to Backlog

tf2b Fork of mercari/tfnotify tf2b parses Terraform commands' execution result and applies it to an arbitrary template and then notifies it to Backlog

Oct 15, 2021
OS Command Execution and Gets the output

OS Command Execution and Gets the output https://stackoverflow.com/questions/24095661/os-exec-sudo-command-in-go/24095983#24095983 https://stackoverfl

Dec 24, 2021
Virtualization system for remote code execution.

Delta CLI Command line application for executing source code inside of a container. Build: go build -o ./delta-cli ./main.go Usage: ./delta-cli <path

Nov 20, 2021
cTRL: a server for remote execution of pending tasks and commands in real time

Документация на русском: https://github.com/eltaline/ctrl/blob/master/README-RUS.md cTRL is a server written in Go language that uses a modified versi

Mar 3, 2022
TUI process monitor written in Go
TUI process monitor written in Go

pst This is TUI process monitor written in Go. Features Monitor process's list, info, tree, open files, Kill process Support OS Mac Linux Requirements

Nov 25, 2022
Yikes is a cli-application to simplify the process to maintaining a list of tasks to complete.

yikes Yikes is a cli-application to simplify the process to maintaining a list of tasks to complete. It also has commands to help store random notes a

Oct 7, 2021
A tool to enumerate all the command-line arguments used to start a Linux process written in Go.
A tool to enumerate all the command-line arguments used to start a Linux process written in Go.

ranwith A tool to enumerate all the command-line arguments used to start a Linux process written in Go. ranwith uses the Linux /proc directory to obta

Jun 30, 2022
🧨 Interactive Process Killer CLI made with Go!
🧨 Interactive Process Killer CLI made with Go!

proc-manager is an interactive CLI to kill processes made with Go, currently supports Linux, Mac OS X, Solaris, and Windows.

Dec 2, 2022
Os-signal-lab - Experiments with os signals and process trees

os-signal-lab Experiments with os signals and process trees There's a lot to rea

Feb 5, 2022
Rule engine implementation in Golang
Rule engine implementation in Golang

"Gopher Holds The Rules" Grule-Rule-Engine import "github.com/hyperjumptech/grule-rule-engine" Rule Engine for Go Grule is a Rule Engine library for t

Jan 3, 2023
User interface engine and widget library for Ebiten
User interface engine and widget library for Ebiten

Ebiten UI A user interface engine and widget library for Ebiten Ebiten UI is an extension to Ebiten that provides an engine to render a complete user

Nov 5, 2022
Simple trie based auto-completion engine implementation in golang.
Simple trie based auto-completion engine implementation in golang.

Simple auto-complete engine implementation in golang. Quick start $ git clone https://github.com/benbarron/trie-auto-completion-engine $ cd trie-auto-

Jul 12, 2022
Vaku is a CLI and API for running path- and folder-based operations on the Vault Key/Value secrets engine.
Vaku is a CLI and API for running path- and folder-based operations on the Vault Key/Value secrets engine.

Vaku Vaku is a CLI and API for running path- and folder-based operations on the Vault Key/Value secrets engine. Vaku extends the existing Vault CLI an

Nov 28, 2022
vkectl is a tool to manage VKE(VolcanoEngine Kubernetes Engine) resources through a CLI

vkectl Introduction vkectl is a tool to manage VKE(VolcanoEngine Kubernetes Engine) resources through a CLI(Command Line Interface). It is written in

Aug 26, 2022
µTask is an automation engine that models and executes business processes declared in yaml. ✏️📋
µTask is an automation engine that models and executes business processes declared in yaml. ✏️📋

µTask, the Lightweight Automation Engine µTask is an automation engine built for the cloud. It is: simple to operate: only a postgres DB is required s

Dec 29, 2022
Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

Cadence Visit cadenceworkflow.io to learn about Cadence. This repo contains the source code of the Cadence server. To implement workflows, activities

Jan 9, 2023
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Nov 10, 2022
Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Apr 21, 2022
An opinionated productive web framework that helps scaling business easier.
An opinionated productive web framework that helps scaling business easier.

appy An opinionated productive web framework that helps scaling business easier, i.e. focus on monolith first, only move to microservices with GRPC la

Nov 4, 2022