Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr

Build Status Go Report Card GoDoc

expr logo

Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not limited to, booleans). It is designed for simplicity, speed and safety.

The purpose of the package is to allow users to use expressions inside configuration for more complex logic. It is a perfect candidate for the foundation of a business rule engine. The idea is to let configure things in a dynamic way without recompile of a program:

# Get the special price if
user.Group in ["good_customers", "collaborator"]

# Promote article to the homepage when
len(article.Comments) > 100 and article.Category not in ["misc"]

# Send an alert when
product.Stock < 15

Features

  • Seamless integration with Go (no need to redefine types)
  • Static typing (example).
    out, err := expr.Compile(`name + age`)
    // err: invalid operation + (mismatched types string and int)
    // | name + age
    // | .....^
  • User-friendly error messages.
  • Reasonable set of basic operators.
  • Builtins all, none, any, one, filter, map.
    all(Tweets, {.Size <= 280})
  • Fast (benchmarks): uses bytecode virtual machine and optimizing compiler.

Install

go get github.com/antonmedv/expr

Documentation

Expr Code Editor

Expr Code Editor

Also, I have an embeddable code editor written in JavaScript which allows editing expressions with syntax highlighting and autocomplete based on your types declaration.

Learn more →

Examples

Play Online

package main

import (
	"fmt"
	"github.com/antonmedv/expr"
)

func main() {
	env := map[string]interface{}{
		"greet":   "Hello, %v!",
		"names":   []string{"world", "you"},
		"sprintf": fmt.Sprintf,
	}

	code := `sprintf(greet, names[0])`

	program, err := expr.Compile(code, expr.Env(env))
	if err != nil {
		panic(err)
	}

	output, err := expr.Run(program, env)
	if err != nil {
		panic(err)
	}

	fmt.Println(output)
}

Play Online

package main

import (
	"fmt"
	"github.com/antonmedv/expr"
)

type Tweet struct {
	Len int
}

type Env struct {
	Tweets []Tweet
}

func main() {
	code := `all(Tweets, {.Len <= 240})`

	program, err := expr.Compile(code, expr.Env(Env{}))
	if err != nil {
		panic(err)
	}

	env := Env{
		Tweets: []Tweet{{42}, {98}, {69}},
	}
	output, err := expr.Run(program, env)
	if err != nil {
		panic(err)
	}

	fmt.Println(output)
}

Contributing

Expr consist of a few packages for parsing source code to AST, type checking AST, compiling to bytecode and VM for running bytecode program.

Also expr provides powerful tool exe for debugging. It has interactive terminal debugger for our bytecode virtual machine.

debugger

Who is using Expr?

  • Aviasales Aviasales are actively using Expr for different parts of the search engine.
  • Argo Argo Rollouts - Progressive Delivery for Kubernetes.
  • Argo Argo Workflows - The workflow engine for KubernetesOverview.
  • CrowdSec Crowdsec - A security automation tool.
  • Mystery Minds uses Expr to allow easy yet powerful customization of its matching algorithm.

Add your company too

License

MIT

Owner
Anton Medvedev
curl medv.io
Anton Medvedev
Comments
  • `integer divide by zero` detection broken on master branch

    `integer divide by zero` detection broken on master branch

    I just updated the expr to latest commit on @master and one of the my unit test is failed. Here is a simple, reproducible code snippet:

    package main
    
    import (
    	"fmt"
    	"github.com/antonmedv/expr"
    )
    
    func main() {
    	env := map[string]uint64{
    		"foo":  7,
    		"bar":  0,
    	}
    
    	code := `(foo / bar) < 10`
    
    	program, err := expr.Compile(code, expr.Env(env))
    	if err != nil {
    		panic(err)
    	}
    
    	output, err := expr.Run(program, env)
    	if err != nil {
    		panic(err)
    	}
    
    	fmt.Println(output)
    }
    

    I was expecting the following error:

    panic: runtime error: integer divide by zero (1:6)
     | (foo / bar) < 10
     | .....^
    

    Works as expected: v1.9.0 Broken (tested at): v1.9.1-0.20221106120435-3d4c21954310

    If bar is 0 in the following expression: (foo / bar) < 10, should be resulting an error.

  • How to access pointer value?

    How to access pointer value?

    I have a usecase where I want to distinguish 0 and nil so I'm trying to pass *int to expr. Below is code I tried and it doesn't work as I had expected. How can get the value of pointer? I want to write expression like *num==1 or num.Value==1 but neither seems to work

    package main
    
    import (
    	"fmt"
    	"github.com/antonmedv/expr"
    )
    
    func main() {
            num := 1
    	env := map[string]interface{}{
    		"num": &num,
    	}
    
    	
    	p1, _ := expr.Compile(`num!=nil`, expr.Env(env))
    	o1, _ := expr.Run(p1, env)
    	fmt.Println(o1) // true
    	
    	p2, _ := expr.Compile(`num==1`, expr.Env(env))
    	o2, _ := expr.Run(p2, env)
    	fmt.Println(o2) // false
    	
    	p3, _ := expr.Compile(`num.Value==1`, expr.Env(env))
    	o3, _ := expr.Run(p3, env)
    	fmt.Println(o3) // nil
    }
    
  • Sandboxing expression

    Sandboxing expression

    We want to be able to sandbox expression evaluation, because some users might submit time, memory, or CPU intensive expression, either accidentally, or as a intentional attack of code that uses expr, example

    map(0..10000, {sprig.genPrivateKey('rsa')}
    
  • A new type named set(contain no repeated value) different from the built-in type map

    A new type named set(contain no repeated value) different from the built-in type map

    Thanks for your contribution, this library help me a lot. Now, I receive a requirement from colleagues, he needs a type "set". The built-in type "map" all contains repeated value, but I want a type that like "map", but without repeating elements, just like type “set” in java, how do i add a new type “set” in this “expr”. @antonmedv

    the new type "set" equal to "map[string]struct{}" in golang

  • Parser error on 386 arch by int overflows

    Parser error on 386 arch by int overflows

    Windows 7 32-bit go1.11.2 windows/386 Trying to do:

    package main
    
    import (
    	"github.com/antonmedv/expr"
    )
    
    func main() {
    	p, err := expr.Compile("1+2") 
    }
    
    

    Build fails with multiple errors:

    Installation failed: # github.com/antonmedv/expr/parser/gen
    ..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2231: constant 4228579072 overflows int
    ..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2249: constant 4228579072 overflows int
    ..\..\..\..\github.com\antonmedv\expr\parser\gen\expr_parser.go:2300: constant 4228579072 overflows int
    
    
  • fast function

    fast function

    I utilized the fast function capability. It still uses reflection. I am trying to avoid the load of reflection. Is there a Fetcher type facility for function in an env struct, to avoid reflection? Is there an alternate way? Is it possible to pre-register a function with its signature to avoid reflection?

    Thank you.

  • JSON marshaling of Program is incomplete

    JSON marshaling of Program is incomplete

    The documentation states that marshaling and unmarshaling of a program is supported. However, the following test fails:

    func Test_marshalRegexp(t *testing.T) {
    	prog, err := expr.Compile(`"hello" matches "h.*"`)
    	if !assert.NoError(t, err) {
    		t.FailNow()
    	}
    
    	marshaled, err := json.Marshal(prog)
    	if !assert.NoError(t, err) {
    		t.FailNow()
    	}
    
    	unmarshaledProgram := &vm.Program{}
    	err = json.Unmarshal(marshaled, unmarshaledProgram)
    	if !assert.NoError(t, err) {
    		t.FailNow()
    	}
    
    	output, err := expr.Run(unmarshaledProgram, nil)
    	if !assert.NoError(t, err) {
    		t.FailNow()
    	}
    	assert.Equal(t, true, output)
    }
    
  • Possible issue with calculation

    Possible issue with calculation

    code:= 1+1/(4+5)*(1*5)+3.3+3.3+3.3+3.3/3.3*3.3

    When I run this code in excel, I get 14.7555. When I run it in expr, I get 14.2.

    What accounts for the difference?

    however this works: 1.0+1/(4.0+5)*(1*5.0)+3.3+3.3+3.3+3.3/3.3*3.3 yields 14.7555

    It looks like you have to marshal the integers into floats by adding .0

    Thank you.

  • Allow renaming struct fields using struct tags

    Allow renaming struct fields using struct tags

    This PR allows renaming struct fields by adding struct tags. For example, the expession lowercaseField + OtherField can be checked and evaluated with the following struct as env:

    type Env struct {
    	UppercaseField int `expr:"lowercaseField"`
    	X              int `expr:"OtherField"`
    }
    

    Closes #13. I drive-by fixed two minor errors in file/source_test.go and gofmt reformatted a few unrelated lines, I hope that's okay for the PR, otherwise I'll revert the unrelated changes.

  • Another go.mod v2 issue?

    Another go.mod v2 issue?

    We have been using v1.1.4 of the 'expr' package, which is great, by the way. We finally updated to GO 1.13 and at the same time started migrating to GO modules.

    However, when upgrading, our process is failing because of a missing "/v2" in your go.mod module path.

    $ go get github.com/antonmedv/[email protected]
    go: finding github.com v2.1.1
    go: finding github.com/antonmedv v2.1.1
    go: finding github.com/antonmedv/expr v2.1.1
    go: finding github.com/antonmedv/expr v2.1.1
    go get github.com/antonmedv/[email protected]: github.com/antonmedv/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2
    

    I was able to download the source and modify the "go.mod" file to be:

    $ cat go.mod
    module github.com/antonmedv/expr/v2
    
    go 1.12
    
    require (
    	github.com/antlr/antlr4 v0.0.0-20190518164840-edae2a1c9b4b
    	github.com/gdamore/tcell v1.1.2
    	github.com/rivo/tview v0.0.0-20190515161233-bd836ef13b4b
    	github.com/sanity-io/litter v1.1.0
    	github.com/stretchr/testify v1.3.0
    	golang.org/x/text v0.3.2
    )
    

    This helped me get past the above error.

    Is there something I am doing wrong? Or is this an issue with the "expr" module. I am new to "go modules" and dealing with the learning curve.

  • Performance of function calls

    Performance of function calls

    Hi @antonmedv,

    Just wondering whether there is something to optimise calling functions? I am testing out vs GoEvaluate and Expr is fastest except for calling functions. I am not sure the reason but I am guessing it may be due to GoEvaluate only defining functions on the compile step. In the examples below I am just doing a simple addition (float64 + float64) in a function.

    I got some performance improvement by using interface{} for the arguments and return value (BenchmarkFuncExprFast), but it still takes twice the time.

    BenchmarkFuncExpr-12                   	 1428219	       881 ns/op	     208 B/op	       8 allocs/op
    BenchmarkFuncExprFast-12               	 1476798	       680 ns/op	     136 B/op	       7 allocs/op
    BenchmarkFuncGoEvaluate-12             	 3650031	       332 ns/op	      88 B/op	       4 allocs/op
    
    func BenchmarkFuncExprFast(b *testing.B) {
    
    	env := Env{
    		"val1": 1.0,
    		"val2": 2.0,
    		"add": func(args ...interface{}) interface{} {
    			return args[0].(float64) + args[1].(float64)
    		},
    	}
    
    	program, err := expr.Compile(`add(val1, val2)`, expr.Env(env))
    	if err != nil {
    		b.Error(err)
    		b.FailNow()
    	}
    
    	b.ResetTimer()
    
    	for i := 0; i < b.N; i++ {
    		val, err := expr.Run(program, env)
    		if err != nil {
    			b.Error(err)
    		}
    		if val != 3.0 {
    			b.Errorf("Val %v not 3", val)
    		}
    	}
    
    }
    
  • Why is the function call speed of expr very slow

    Why is the function call speed of expr very slow

    I did a benchmark test on several expression parsing libraries, including function calls and injecting structs, and I found that expr's function calls were much slower compared to the other products. I want to figure out the reason for this.

    $ go test -bench=. -benchtime=10s
    goos: darwin
    goarch: amd64
    pkg: github.com/antonmedv/golang-expression-evaluation-comparison
    cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Benchmark_bexpr-12                 	 5676273	      2089 ns/op
    Benchmark_celgo-12                 	77683569	       153.0 ns/op
    Benchmark_celgo_startswith-12      	42621015	       278.7 ns/op
    Benchmark_celgo_funccall-12        	69535796	       172.7 ns/op
    Benchmark_celgo_struct-12          	16101632	       748.2 ns/op
    Benchmark_evalfilter-12            	 7971537	      1508 ns/op
    Benchmark_expr-12                  	93287458	       126.6 ns/op
    Benchmark_expr_startswith-12       	48665090	       246.6 ns/op
    Benchmark_expr_funccall-12         	22060058	       544.6 ns/op
    Benchmark_expr_struct-12           	39742920	       300.1 ns/op
    Benchmark_goja-12                  	41479744	       286.4 ns/op
    Benchmark_govaluate-12             	55276390	       213.1 ns/op
    Benchmark_govaluate_funccall-12    	87888637	       125.8 ns/op
    Benchmark_govaluate_struct-12      	19006689	       629.2 ns/op
    Benchmark_gval-12                  	20624302	       579.2 ns/op
    Benchmark_otto-12                  	19031895	       630.9 ns/op
    Benchmark_starlark-12              	 2717073	      4394 ns/op
    PASS
    ok  	github.com/antonmedv/golang-expression-evaluation-comparison	215.618s
    

    test code

    func Benchmark_expr_funccall(b *testing.B) {
    	params := map[string]interface{}{
    		"hello": func(str string) string { return "hello " + str },
    	}
    
    	program, err := expr.Compile(`hello("world")`)
    	if err != nil {
    		b.Fatal(err)
    	}
    
    	var out interface{}
    
    	b.ResetTimer()
    	for n := 0; n < b.N; n++ {
    		out, err = expr.Run(program, params)
    	}
    	b.StopTimer()
    
    	if err != nil {
    		b.Fatal(err)
    	}
    	if out.(string) != "hello world" {
    		b.Fail()
    	}
    }
    
  • Improve testing, remove generated code and add duration operators

    Improve testing, remove generated code and add duration operators

    This PR should add a number of improvements:

    1. Table tests are now run as separate tests. The reason for this is how common Go tests are structured as well as human-readability in case of multiple failing tests: Each failing test is reported and not only a single one. This also makes running tests in parallel easier. On my machine, tests still run in less than 0.09s.
    2. Support for Go's time.Duration is now available. This includes many math operations and comparisons. A required change for this was dropping generated helper functions. This is justified by the fact that time.Duration-support requires custom logic. For example:
    • When performing addition with time.Time, we need to use methods from time.Time.
    • When combined with numeric types, we treat time.Duration similarly to int64.
    • When performing division, e.g., time.Hour / 6, we expect time.Duration as result type.

    The new approach treats all uint/int-types as int. Code generation was removed and replaced by custom logic for each operation.

    For further information, see commit messages.

  • Ability to pass custom Func to CallNode

    Ability to pass custom Func to CallNode

    Carry: https://github.com/antonmedv/expr/issues/277

    Motivation

    In the current implementation, you can not pass a function to CallNode directly. It’s the only option to pass a func from env; which you need to set Callee field.

    Use Case

    divFn := func(left, right float64) float64 {
    	if right == 0 {
    		panic("integer divide by zero")
    	}
    	return left / right
    }
    
    ast.Patch(node, &ast.CallNode{
    	Func:    divFn,
    	Arguments: []ast.Node{n.Left, n.Right},
    })
    
  • Provide websssembly output

    Provide websssembly output

    exprs implementation is great. It would be great to provide a websssembly compiler that would enable the execution of compiled expressions anywhere that can run webasm

  • Add builtin math functions

    Add builtin math functions

    • abs(x)
    • acos(x)
    • acosh(x)
    • asin(x)
    • asinh(x)
    • atan(x)
    • atan2(y, x)
    • atanh(x)
    • cbrt(x)
    • ceil(x)
    • cos(x)
    • cosh(x)
    • dim(x, y)
    • erf(x)
    • erfc(x)
    • erfcinv(x)
    • erfinv(x)
    • exp(x)
    • exp2(x)
    • expm1(x)
    • fma(x, y, z)
    • floor(x)
    • gamma(x)
    • hypot(p, q)
    • ilogb(x) int
    • j0(x)
    • j1(x)
    • jn(n int, x)
    • ldexp(frac, exp int)
    • log(x)
    • log10(x)
    • log1p(x)
    • log2(x)
    • logb(x)
    • max(x, y)
    • min(x, y)
    • mod(x, y)
    • pow(x, y)
    • pow10(n int)
    • remainder(x, y)
    • round(x)
    • roundtoeven(x)
    • sin(x)
    • sinh(x)
    • sqrt(x)
    • tan(x)
    • tanh(x)
    • trunc(x)
    • y0(x)
    • y1(x)
    • yn(n int, x)
Expression evaluation in golang
Expression evaluation in golang

Gval Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions. Evaluate Gval can evaluate expressio

Dec 27, 2022
Arbitrary expression evaluation for golang

govaluate Provides support for evaluating arbitrary C-like artithmetic/string expressions. Why can't you just write these expressions in code? Sometim

Jan 2, 2023
Mathematical expression parsing and calculation engine library. 数学表达式解析计算引擎库

Math-Engine 使用 Go 实现的数学表达式解析计算引擎库,它小巧,无任何依赖,具有扩展性(比如可以注册自己的函数到引擎中),比较完整的完成了数学表达式解析执行,包括词法分析、语法分析、构建AST、运行。 go get -u github.com/dengsgo/math-engine 能够

Jan 3, 2023
Logexp - Logical expression compiler for golang

Logical Expression Compiler Functions: - Compile(exp string) - Match(text string

Jan 24, 2022
Suan - Mathematical expression calculation tool

suan Suan( 算 ) is a CLI tool to calculate given mathematical expression. Current

Feb 14, 2022
ECMAScript/JavaScript engine in pure Go

goja ECMAScript 5.1(+) implementation in Go. Goja is an implementation of ECMAScript 5.1 in pure Go with emphasis on standard compliance and performan

Jan 1, 2023
A fast script language for Go
A fast script language for Go

The Tengo Language Tengo is a small, dynamic, fast, secure script language for Go. Tengo is fast and secure because it's compiled/executed as bytecode

Dec 30, 2022
Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. https://vlang.io
Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. https://vlang.io

The V Programming Language vlang.io | Docs | Changelog | Speed | Contributing & compiler design Key Features of V Simplicity: the language can be lear

Jan 4, 2023
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Dec 30, 2022
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Dec 30, 2022
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Jan 3, 2023
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Dec 24, 2022
Go-turing-i2c-cmdline - Controlling the i2c management bus of the turing pi with i2c works fine

go-turing-i2c-cmdline What is it? Controlling the i2c management bus of the turi

Jan 24, 2022
Simple expression evaluation engine for Go

??️ chili Currently in development, Unstable (API may change in future) Simple expression evaluation engine. Expression is one liner that evalutes int

Nov 8, 2022
Expression evaluation in golang
Expression evaluation in golang

Gval Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions. Evaluate Gval can evaluate expressio

Dec 27, 2022
Expression evaluation in golang
Expression evaluation in golang

Gval Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions. Evaluate Gval can evaluate expressio

Dec 27, 2022
Arbitrary expression evaluation for golang

govaluate Provides support for evaluating arbitrary C-like artithmetic/string expressions. Why can't you just write these expressions in code? Sometim

Jan 2, 2023
The Dual-Stack Dynamic DNS client, the world's first dynamic DNS client built for IPv6.

dsddns DsDDNS is the Dual-Stack Dynamic DNS client. A dynamic DNS client keeps your DNS records in sync with the IP addresses associated with your hom

Sep 27, 2022
An implementation of Neural Turing Machines
An implementation of Neural Turing Machines

Neural Turing Machines Package ntm implements the Neural Turing Machine architecture as described in A.Graves, G. Wayne, and I. Danihelka. arXiv prepr

Sep 13, 2022
A complete Liquid template engine in Go
A complete Liquid template engine in Go

Liquid Template Parser liquid is a pure Go implementation of Shopify Liquid templates. It was developed for use in the Gojekyll port of the Jekyll sta

Dec 15, 2022