Arbitrary expression evaluation for golang

govaluate

Build Status Godoc Go Report Card Gocover

Provides support for evaluating arbitrary C-like artithmetic/string expressions.

Why can't you just write these expressions in code?

Sometimes, you can't know ahead-of-time what an expression will look like, or you want those expressions to be configurable. Perhaps you've got a set of data running through your application, and you want to allow your users to specify some validations to run on it before committing it to a database. Or maybe you've written a monitoring framework which is capable of gathering a bunch of metrics, then evaluating a few expressions to see if any metrics should be alerted upon, but the conditions for alerting are different for each monitor.

A lot of people wind up writing their own half-baked style of evaluation language that fits their needs, but isn't complete. Or they wind up baking the expression into the actual executable, even if they know it's subject to change. These strategies may work, but they take time to implement, time for users to learn, and induce technical debt as requirements change. This library is meant to cover all the normal C-like expressions, so that you don't have to reinvent one of the oldest wheels on a computer.

How do I use it?

You create a new EvaluableExpression, then call "Evaluate" on it.

	expression, err := govaluate.NewEvaluableExpression("10 > 0");
	result, err := expression.Evaluate(nil);
	// result is now set to "true", the bool value.

Cool, but how about with parameters?

	expression, err := govaluate.NewEvaluableExpression("foo > 0");

	parameters := make(map[string]interface{}, 8)
	parameters["foo"] = -1;

	result, err := expression.Evaluate(parameters);
	// result is now set to "false", the bool value.

That's cool, but we can almost certainly have done all that in code. What about a complex use case that involves some math?

	expression, err := govaluate.NewEvaluableExpression("(requests_made * requests_succeeded / 100) >= 90");

	parameters := make(map[string]interface{}, 8)
	parameters["requests_made"] = 100;
	parameters["requests_succeeded"] = 80;

	result, err := expression.Evaluate(parameters);
	// result is now set to "false", the bool value.

Or maybe you want to check the status of an alive check ("smoketest") page, which will be a string?

	expression, err := govaluate.NewEvaluableExpression("http_response_body == 'service is ok'");

	parameters := make(map[string]interface{}, 8)
	parameters["http_response_body"] = "service is ok";

	result, err := expression.Evaluate(parameters);
	// result is now set to "true", the bool value.

These examples have all returned boolean values, but it's equally possible to return numeric ones.

	expression, err := govaluate.NewEvaluableExpression("(mem_used / total_mem) * 100");

	parameters := make(map[string]interface{}, 8)
	parameters["total_mem"] = 1024;
	parameters["mem_used"] = 512;

	result, err := expression.Evaluate(parameters);
	// result is now set to "50.0", the float64 value.

You can also do date parsing, though the formats are somewhat limited. Stick to RF3339, ISO8061, unix date, or ruby date formats. If you're having trouble getting a date string to parse, check the list of formats actually used: parsing.go:248.

	expression, err := govaluate.NewEvaluableExpression("'2014-01-02' > '2014-01-01 23:59:59'");
	result, err := expression.Evaluate(nil);

	// result is now set to true

Expressions are parsed once, and can be re-used multiple times. Parsing is the compute-intensive phase of the process, so if you intend to use the same expression with different parameters, just parse it once. Like so;

	expression, err := govaluate.NewEvaluableExpression("response_time <= 100");
	parameters := make(map[string]interface{}, 8)

	for {
		parameters["response_time"] = pingSomething();
		result, err := expression.Evaluate(parameters)
	}

The normal C-standard order of operators is respected. When writing an expression, be sure that you either order the operators correctly, or use parenthesis to clarify which portions of an expression should be run first.

Escaping characters

Sometimes you'll have parameters that have spaces, slashes, pluses, ampersands or some other character that this library interprets as something special. For example, the following expression will not act as one might expect:

"response-time < 100"

As written, the library will parse it as "[response] minus [time] is less than 100". In reality, "response-time" is meant to be one variable that just happens to have a dash in it.

There are two ways to work around this. First, you can escape the entire parameter name:

"[response-time] < 100"

Or you can use backslashes to escape only the minus sign.

"response\\-time < 100"

Backslashes can be used anywhere in an expression to escape the very next character. Square bracketed parameter names can be used instead of plain parameter names at any time.

Functions

You may have cases where you want to call a function on a parameter during execution of the expression. Perhaps you want to aggregate some set of data, but don't know the exact aggregation you want to use until you're writing the expression itself. Or maybe you have a mathematical operation you want to perform, for which there is no operator; like log or tan or sqrt. For cases like this, you can provide a map of functions to NewEvaluableExpressionWithFunctions, which will then be able to use them during execution. For instance;

	functions := map[string]govaluate.ExpressionFunction {
		"strlen": func(args ...interface{}) (interface{}, error) {
			length := len(args[0].(string))
			return (float64)(length), nil
		},
	}

	expString := "strlen('someReallyLongInputString') <= 16"
	expression, _ := govaluate.NewEvaluableExpressionWithFunctions(expString, functions)

	result, _ := expression.Evaluate(nil)
	// result is now "false", the boolean value

Functions can accept any number of arguments, correctly handles nested functions, and arguments can be of any type (even if none of this library's operators support evaluation of that type). For instance, each of these usages of functions in an expression are valid (assuming that the appropriate functions and parameters are given):

"sqrt(x1 ** y1, x2 ** y2)"
"max(someValue, abs(anotherValue), 10 * lastValue)"

Functions cannot be passed as parameters, they must be known at the time when the expression is parsed, and are unchangeable after parsing.

Accessors

If you have structs in your parameters, you can access their fields and methods in the usual way. For instance, given a struct that has a method "Echo", present in the parameters as foo, the following is valid:

"foo.Echo('hello world')"

Fields are accessed in a similar way. Assuming foo has a field called "Length":

"foo.Length > 9000"

Accessors can be nested to any depth, like the following

"foo.Bar.Baz.SomeFunction()"

However it is not currently supported to access values in maps. So the following will not work

"foo.SomeMap['key']"

This may be convenient, but note that using accessors involves a lot of reflection. This makes the expression about four times slower than just using a parameter (consult the benchmarks for more precise measurements on your system). If at all reasonable, the author recommends extracting the values you care about into a parameter map beforehand, or defining a struct that implements the Parameters interface, and which grabs fields as required. If there are functions you want to use, it's better to pass them as expression functions (see the above section). These approaches use no reflection, and are designed to be fast and clean.

What operators and types does this support?

  • Modifiers: + - / * & | ^ ** % >> <<
  • Comparators: > >= < <= == != =~ !~
  • Logical ops: || &&
  • Numeric constants, as 64-bit floating point (12345.678)
  • String constants (single quotes: 'foobar')
  • Date constants (single quotes, using any permutation of RFC3339, ISO8601, ruby date, or unix date; date parsing is automatically tried with any string constant)
  • Boolean constants: true false
  • Parenthesis to control order of evaluation ( )
  • Arrays (anything separated by , within parenthesis: (1, 2, 'foo'))
  • Prefixes: ! - ~
  • Ternary conditional: ? :
  • Null coalescence: ??

See MANUAL.md for exacting details on what types each operator supports.

Types

Some operators don't make sense when used with some types. For instance, what does it mean to get the modulo of a string? What happens if you check to see if two numbers are logically AND'ed together?

Everyone has a different intuition about the answers to these questions. To prevent confusion, this library will refuse to operate upon types for which there is not an unambiguous meaning for the operation. See MANUAL.md for details about what operators are valid for which types.

Benchmarks

If you're concerned about the overhead of this library, a good range of benchmarks are built into this repo. You can run them with go test -bench=.. The library is built with an eye towards being quick, but has not been aggressively profiled and optimized. For most applications, though, it is completely fine.

For a very rough idea of performance, here are the results output from a benchmark run on a 3rd-gen Macbook Pro (Linux Mint 17.1).

BenchmarkSingleParse-12                          1000000              1382 ns/op
BenchmarkSimpleParse-12                           200000             10771 ns/op
BenchmarkFullParse-12                              30000             49383 ns/op
BenchmarkEvaluationSingle-12                    50000000                30.1 ns/op
BenchmarkEvaluationNumericLiteral-12            10000000               119 ns/op
BenchmarkEvaluationLiteralModifiers-12          10000000               236 ns/op
BenchmarkEvaluationParameters-12                 5000000               260 ns/op
BenchmarkEvaluationParametersModifiers-12        3000000               547 ns/op
BenchmarkComplexExpression-12                    2000000               963 ns/op
BenchmarkRegexExpression-12                       100000             20357 ns/op
BenchmarkConstantRegexExpression-12              1000000              1392 ns/op
ok

API Breaks

While this library has very few cases which will ever result in an API break, it can (and has) happened. If you are using this in production, vendor the commit you've tested against, or use gopkg.in to redirect your import (e.g., import "gopkg.in/Knetic/govaluate.v2"). Master branch (while infrequent) may at some point contain API breaking changes, and the author will have no way to communicate these to downstreams, other than creating a new major release.

Releases will explicitly state when an API break happens, and if they do not specify an API break it should be safe to upgrade.

License

This project is licensed under the MIT general use license. You're free to integrate, fork, and play with this code as you feel fit without consulting the author, as long as you provide proper credit to the author in your works.

Owner
George Lester
If something is hard, that's a pretty good reason to solve it in a way that makes it easy in the future.
George Lester
Comments
  • Can I use a struct method (or have an object pointer as the parameter) in functions?

    Can I use a struct method (or have an object pointer as the parameter) in functions?

    Hi, I have a struct like testUser as below.

    type testUser struct {
    	name string
    	domain string
    }
    
    func newTestUser(name string, domain string) *testUser {
    	u := testUser{}
    	u.name = name
    	u.domain = domain
    	return &u
    }
    
    func (u *testUser) getAttribute(attributeName string) string {
    	ru := reflect.ValueOf(u)
    	f := reflect.Indirect(ru).FieldByName(attributeName)
    	return f.String()
    }
    
    func getAttribute2(u *testUser, attributeName string) string {
    	ru := reflect.ValueOf(u)
    	f := reflect.Indirect(ru).FieldByName(attributeName)
    	return f.String()
    }
    
    func getAttribute3(userName string, attributeName string) string {
    	u := getUserObjectByName(userName) // for example using a map
    	ru := reflect.ValueOf(u)
    	f := reflect.Indirect(ru).FieldByName(attributeName)
    	return f.String()
    }
    
    alice := newTestUser("alice", "domain1")
    

    I'd like to make the properties of testUser object dynamically parsed in govaluate, the object is an input parameter. I don't know if this is possible.

    1. The best expectation is: expression = alice.name, and use alice object as a parameter. If this is not possible:
    2. I define a func (u *testUser) getAttribute(attributeName string) string function as above. expression = alice.getAttribute("name"), and use alice object as a parameter. If this is either not possible:
    3. I define a global func getAttribute2(u *testUser, attributeName string) string function as above. expression = getAttribute2(alice, "name"), and use alice object as a parameter.
    4. The last way to to define func getAttribute3(userName string, attributeName string) string. expression = getAttribute2("alice", "name"), and use alice name as a string parameter. I know this way works. But it is inconvenient because I need to implement a get-object-by-name function via a map. So I don't want this way.

    So my priority is: 1. > 2. > 3. > 4. Do any of the above ideas 1, 2, 3 work in govaluate? Thanks!

  • Test if expression will yield a numeric or boolean

    Test if expression will yield a numeric or boolean

    Would it be possible to add a function which would tell you whether or not a given expression yielded a boolean or numeric value without submitting any parameter values?

  • Odd Behavior with order dependent operators (`-` and `/`)

    Odd Behavior with order dependent operators (`-` and `/`)

    Take these two test for example

    2 / 6 / 3 and 2 - 6 - 10 - 2, simple enough right?

    Running those same expressions through nodejs, ruby, and go gives different results then this evaluator.

    Node:

    > 2 / 6 / 3
    0.1111111111111111
    > 2 - 6 - 10 - 2
    -16
    

    Ruby:

    2.0.0-p645 :002 > 2.0 / 6.0 / 3.0
     => 0.1111111111111111
    2.0.0-p645 :003 > 2 - 6 - 10 - 2
     => -16
    

    Go:

    /////////////////////////////
    // main.go
    /////////////////////////////
    package main
    
    import (
      "fmt"
    )
    
    func main() {
      t1 := 2.0 / 6.0 / 3.0
      fmt.Println(t1)
      t2 := 2 - 6 - 10 - 2
      fmt.Println(t2)
    }
    /////////////////////////////
    /////////////////////////////
    
    $ go run main.go
    0.1111111111111111
    -16
    
    EvaluationTest{
      Name:     "Incorrect divide behavior",
      Input:    "2 / 6 / 3",
      Expected: 0.1111111111111111,
    },
    EvaluationTest{
      Name:     "Incorrect subtract behavior",
      Input:    "2 - 6 - 10 - 2",
      Expected: -16.0,
    }
    
    evaluation_test.go:877: Test 'Incorrect divide behavior' failed
    evaluation_test.go:878: Evaluation result '1' does not match expected: '0.1111111111111111'
    evaluation_test.go:877: Test 'Incorrect subtract behavior' failed
    evaluation_test.go:878: Evaluation result '4' does not match expected: '-16'
    

    The issue is the parser is moving right -> left NOT left -> right . I tried playing around with swapping the value vs rightValue orders but it gets messy. I'm sure this is a simple fix but I'm still trying to fully understand how the evaluate chain and the token stream all play together in respect to order of evaluation.

    Cheers, W

  • Expression evaluation errors

    Expression evaluation errors

    Hello and thanks for the useful library, but i've got an issue with it.

    I have a quite simple expression: ([X] >= 2887057408 && [X] <= 2887122943) || ([X] >= 168100864 && [X] <= 168118271)

    If i pass X = 2887057409, which satisfies first expression in parenthesis, the expression is for some reason evaluated to FALSE.

    If i pass X = 168100865, which satisfies second part of an expression in parenthesis, i get TRUE.

    If i swap the sub-expressions, then i get the inverted behaviour.

    If i use smaller numbers, like: ([X] >= 0 && [X] <= 10) || ([X] >= 20 && [X] <= 30)

    then the expression is evaluated right in all cases.

    I think the problem is with numbers being converted to float64, however looking at tokens after expression compilation the numbers seem fine: {Kind:13 Value:40} {Kind:7 Value:X} {Kind:10 Value:>=} {Kind:2 Value:2.887057408e+09} {Kind:11 Value:&&} {Kind:7 Value:X} {Kind:10 Value:<=} {Kind:2 Value:2.887122943e+09} {Kind:14 Value:41} {Kind:11 Value:||} {Kind:13 Value:40} {Kind:7 Value:X} {Kind:10 Value:>=} {Kind:2 Value:1.68100864e+08} {Kind:11 Value:&&} {Kind:7 Value:X} {Kind:10 Value:<=} {Kind:2 Value:1.68118271e+08} {Kind:14 Value:41}

    And this does not explain why the behaviour changes if i just reverse the parts before and after || operator...

  • REQUEST - support user defined functions

    REQUEST - support user defined functions

    hi, this library is great. User defined parameters/variables is very welcome. Is there any chance you can add user defined functions as well (ideally nested user defined functions).

    example:

    "(requests_made("5m") * requests_succeeded("5m") / 100) >= 90"

  • Add support for bitwise operators.

    Add support for bitwise operators.

    & (and), | (or), ^ (xor), and ~ (not) are now supported.

    Float64 values are truncated to int64 then converted back.

    The exponent operator is now **.

  • Added functions conjuction

    Added functions conjuction

    Simple syntax for functions conjunction the result if function which before '|>' will be added as last argument to function which is after '|>'

    "foo() |> bar(10) |> woo(20)
    // it is same as
    "woo(20, bar(10, foo()))"
    
  • issues with functions?

    issues with functions?

    I wrote this function:

        functions := map[string]govaluate.ExpressionFunction{
            "now": func(args ...interface{}) (interface{}, error) {
                return float64(time.Now().UnixNano() / 1e6), nil
            },
        }
    

    Executing 1+now() works but now()+1 yields "Unable to plan token kind: MODIFIER".

    More complex arithmetic expressions including now() trigger other errors.

  • Can you help me slim down the library?

    Can you help me slim down the library?

    Thanks for this awesome library! For a special use case I tried to slim it down to work only with float64 as parameters and results, and maybe boolean as interim results. I want to get rid of the interfaces for performance reasons and wanted the result to be an error flag when one of the parameters is an error flag (>= 9.999.999). I started by changing the parameters map to be a map[string]float64 and then tried to hunt down the compiler errors. But I have a hard time doing so. May I ask for some hint?

    Thanks in advance!

  • many IN array feature

    many IN array feature

    Hi,

    First, thanks for the library, its awesome! I have a question on how to use/extent the library to fit my needs. I intend to use it in order to eval questionnaires, while every questionnaire composed of dif questions types. Some are simple, for instance a dates compare, single answer IN array etc.., but how can i implement MANY in ARRAY, e.g. my use selects many choices and there are many possible answers? I checked the library and this expression "(2) IN (1, 2, 3)" works as oppose to "(1,2) IN (1, 2, 3)". I thought about extend it myself but as my expression will be used for many questionnaires i want to cache the right side (1,2,3) into a map to achieve performance... How can it be done?

    thanks!

  • Passing date values in parameters

    Passing date values in parameters

    First of all, thank you so much for your library. It is awesome.

    I am trying to evaluate an expression with a date value set in parameters. I have posted a question on StackOverflow about this: https://stackoverflow.com/questions/44893202/compare-date-values-in-expression-using-govaluate

    Is there a way i can achieve the following:

    package main
    
    import (
        "github.com/Knetic/govaluate"
        "fmt"
    )
    
    func main()  {
        expression, err := govaluate.NewEvaluableExpression("first_name == 'Julian' && emp_id == 302 && birth_date >= '2016-12-12' && birth_date <= '2016-12-25'")
    
        parameters := make(map[string]interface{}, 8)
        parameters["first_name"] = "Julian"
        parameters["emp_id"] = 302
        parameters["birth_date"] = "2016-12-15"
        result, err := expression.Evaluate(parameters)
    
        if err != nil {
            fmt.Println("Error while evaluating the expression")
        }
    
        if result.(bool) {
            fmt.Println("Condition is evaluated to true")
        } else {
            fmt.Println("Condition is evaluated to false")
        }
    }
    

    I a getting this error:

    Error while evaluating the expression
    panic: interface conversion: interface is nil, not bool
    

    I tried doing the following: convert the date literal into float and pass as a parameter.

    dateFloat, _ := strconv.ParseFloat("2016-12-15", 64)
    parameters["birth_date"] = dateFloat
    

    Although, the error disappeared, the result value is false which is logically wrong since "2016-12-15" falls between "2016-12-12" and "2016-12-25".

    Can you suggest me a way to achieve it?

  • Another Bug with nested ternaries

    Another Bug with nested ternaries

    1. The following nested ternary expression fails to evaluate: (44 != 0 ? 44 : 22 != 0 ? 22 : 11) != 0 the reported error was: eval error in boolean expression

    2. A very similar ternary expression evaluates OK: (44 < 33 ? 44 : 22 < 11 ? 22 : 11) != 0 I presume the difference is in the use of reflect.DeepEqual for the != operator and not for the < operator.

    3. Another similar nested ternary expression to case 1. is also OK, despite the fact that it uses the != operator: (44 != 0 ? 33 != 0 ? 33 : 22 : 11) != 0

    These expressions are generated by users of the language translator I am developing. Expression 1. works Ok in C.

    I am very happy I found the govaluate library - it does a great job for the language translator - Thank you.

  • precision lost

    precision lost

    	expression, err := govaluate.NewEvaluableExpression("(0.05 + 0.01) > 0.06")
    	assert.NoError(t, err)
    	result, err := expression.Evaluate(nil)
    	assert.NoError(t, err)
    	assert.Equal(t, false, result.(bool))
    

    Error: Not equal: expected: false actual : true

  • Fix evaluation_test on 32 bit architectures

    Fix evaluation_test on 32 bit architectures

    Before this patch this is the result on i586:

    # github.com/Knetic/govaluate [github.com/Knetic/govaluate.test]
    ./evaluation_test.go:1166:6: constant 2887057409 overflows int
    ./evaluation_test.go:1178:6: constant 2887057409 overflows int
    FAIL	github.com/Knetic/govaluate [build failed]
    
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

May 10, 2022
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

May 10, 2022
Logexp - Logical expression compiler for golang

Logical Expression Compiler Functions: - Compile(exp string) - Match(text string

Jan 24, 2022
Mathematical expression parsing and calculation engine library. 数学表达式解析计算引擎库

Math-Engine 使用 Go 实现的数学表达式解析计算引擎库,它小巧,无任何依赖,具有扩展性(比如可以注册自己的函数到引擎中),比较完整的完成了数学表达式解析执行,包括词法分析、语法分析、构建AST、运行。 go get -u github.com/dengsgo/math-engine 能够

May 14, 2022
Suan - Mathematical expression calculation tool

suan Suan( 算 ) is a CLI tool to calculate given mathematical expression. Current

Feb 14, 2022
Transpiling fortran code to golang code

f4go Example of use > # Install golang > # Compile f4go > go get -u github.com/Konstantin8105/f4go > cd $GOPATH/src/github.com/Konstantin8105/f4go > g

Apr 11, 2022
Golang->Haxe->CPP/CSharp/Java/JavaScript transpiler

TARDIS Go -> Haxe transpiler Haxe -> C++ / C# / Java / JavaScript Project status: a non-working curiosity, development currently on-ice The advent of

Apr 18, 2022
A JavaScript interpreter in Go (golang)

otto -- import "github.com/robertkrimen/otto" Package otto is a JavaScript parser and interpreter written natively in Go. http://godoc.org/github.com/

May 11, 2022
A BASIC interpreter written in golang.
A BASIC interpreter written in golang.

05 PRINT "Index" 10 PRINT "GOBASIC!" 20 PRINT "Limitations" Arrays Line Numbers IF Statement DATA / READ Statements Builtin Functions Types 30 PRINT "

May 5, 2022
PHP bindings for the Go programming language (Golang)

PHP bindings for Go This package implements support for executing PHP scripts, exporting Go variables for use in PHP contexts, attaching Go method rec

May 9, 2022
High-performance PHP-to-Golang IPC bridge

High-performance PHP-to-Golang IPC bridge Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/r

May 7, 2022
High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

May 11, 2022
golang AST matcher

goastch (GO AST matCH) Introduction Inspired by ast matcher. There are four different basic categories of matchers: Node Matchers: Matchers that match

Apr 19, 2022
Scriptable interpreter written in golang
Scriptable interpreter written in golang

Anko Anko is a scriptable interpreter written in Go. (Picture licensed under CC BY-SA 3.0, photo by Ocdp) Usage Example - Embedded package main impor

May 13, 2022
hotbuild - a cross platform hot compilation tool for golang
hotbuild - a cross platform hot compilation tool for golang

hotbuild A cross platform hot compilation tool By monitoring the modification of the project directory file, the recompilation and running are automat

May 9, 2022
The golang tool of the zig compiler automatically compiles different targets according to the GOOS GOARCH environment variable. You need to install zig.

The golang tool of the zig compiler automatically compiles different targets according to the GOOS GOARCH environment variable. You need to install zig.

Apr 24, 2022
Tgo - Test Helpers for Standard Golang Packages

Test Helpers for Standard Golang Packages see example_test.go go test --- FAIL:

Apr 26, 2022
Runcmd - just golang binary that runs commands from url or local file and logs output

runcmd just golang binary that runs commands from url or local file and logs out

Feb 2, 2022
A compiler for the ReCT programming language written in Golang

ReCT-Go-Compiler A compiler for the ReCT programming language written in Golang

Apr 7, 2022