Expression evaluation in golang

Gval

Go Reference Build Status Coverage Status Go Report Card

Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions.

gopher

Evaluate

Gval can evaluate expressions with parameters, arimethetic, logical, and string operations:

It can easily be extended with custom functions or operators:

You can parse gval.Expressions once and re-use them multiple times. Parsing is the compute-intensive phase of the process, so if you intend to use the same expression with different parameters, just parse it once:

The normal Go-standard order of operators is respected. When writing an expression, be sure that you either order the operators correctly, or use parentheses to clarify which portions of an expression should be run first.

Strings, numbers, and booleans can be used like in Go:

Parameter

Variables can be accessed via string literals. They can be used for values with string keys if the parameter is a map[string]interface{} or map[interface{}]interface{} and for fields or methods if the parameter is a struct.

Bracket Selector

Map and array elements and Struct Field can be accessed via [].

Dot Selector

A nested variable with a name containing only letters and underscores can be accessed via a dot selector.

Custom Selector

Parameter names like response-time will be interpreted as response minus time. While gval doesn't support these parameter names directly, you can easily access them via a custom extension like JSON Path:

Jsonpath is also suitable for accessing array elements.

Fields and Methods

If you have structs in your parameters, you can access their fields and methods in the usual way:

It also works if the parameter is a struct directly Hello + World() or if the fields are nested foo.Hello + foo.World()

This may be convenient but note that using accessors on strucs makes the expression about four times slower than just using a parameter (consult the benchmarks for more precise measurements on your system). If there are functions you want to use, it's faster (and probably cleaner) to define them as functions (see the Evaluate section). These approaches use no reflection, and are designed to be fast and clean.

Default Language

The default language is in serveral sub languages like text, arithmetic or propositional logic defined. See Godoc for details. All sub languages are merged into gval.Full which contains the following elements:

  • Modifiers: + - / * & | ^ ** % >> <<
  • Comparators: > >= < <= == != =~ !~
  • Logical ops: || &&
  • Numeric constants, as 64-bit floating point (12345.678)
  • String constants (double quotes: "foobar")
  • Date function 'Date(x)', using any permutation of RFC3339, ISO8601, ruby date, or unix date
  • Boolean constants: true false
  • Parentheses to control order of evaluation ( )
  • Json Arrays : [1, 2, "foo"]
  • Json Objects : {"a":1, "b":2, "c":"foo"}
  • Prefixes: ! - ~
  • Ternary conditional: ? :
  • Null coalescence: ??

Customize

Gval is completly customizable. Every constant, function or operator can be defined separately and existing expression languages can be reused:

For details see Godoc.

Implementing custom selector

In a case you want to provide custom logic for selectors you can implement SelectGVal(ctx context.Context, k string) (interface{}, error) on your struct. Function receives next part of the path and can return any type of var that is again evaluated through standard gval procedures.

Example Custom Selector

External gval Languages

A list of external libraries for gval. Feel free to add your own library.

Performance

The library is built with the intention of being quick but has not been aggressively profiled and optimized. For most applications, though, it is completely fine. If performance is an issue, make sure to create your expression language with all functions, constants and operators only once. Evaluating an expression like gval.Evaluate("expression, const1, func1, func2, ...) creates a new gval.Language everytime it is called and slows execution.

The library comes with a bunch of benchmarks to measure the performance of parsing and evaluating expressions. You can run them with go test -bench=..

For a very rough idea of performance, here are the results from a benchmark run on a Dell Latitude E7470 Win 10 i5-6300U.

BenchmarkGval/const_evaluation-4                               500000000                 3.57 ns/op
BenchmarkGval/const_parsing-4                                    1000000              1144 ns/op
BenchmarkGval/single_parameter_evaluation-4                     10000000               165 ns/op
BenchmarkGval/single_parameter_parsing-4                         1000000              1648 ns/op
BenchmarkGval/parameter_evaluation-4                             5000000               352 ns/op
BenchmarkGval/parameter_parsing-4                                 500000              2773 ns/op
BenchmarkGval/common_evaluation-4                                3000000               434 ns/op
BenchmarkGval/common_parsing-4                                    300000              4419 ns/op
BenchmarkGval/complex_evaluation-4                             100000000                11.6 ns/op
BenchmarkGval/complex_parsing-4                                   100000             17936 ns/op
BenchmarkGval/literal_evaluation-4                             300000000                 3.84 ns/op
BenchmarkGval/literal_parsing-4                                   500000              2559 ns/op
BenchmarkGval/modifier_evaluation-4                            500000000                 3.54 ns/op
BenchmarkGval/modifier_parsing-4                                  500000              3755 ns/op
BenchmarkGval/regex_evaluation-4                                   50000             21347 ns/op
BenchmarkGval/regex_parsing-4                                     200000              6480 ns/op
BenchmarkGval/constant_regex_evaluation-4                        1000000              1000 ns/op
BenchmarkGval/constant_regex_parsing-4                            200000              9417 ns/op
BenchmarkGval/accessors_evaluation-4                             3000000               417 ns/op
BenchmarkGval/accessors_parsing-4                                1000000              1778 ns/op
BenchmarkGval/accessors_method_evaluation-4                      1000000              1931 ns/op
BenchmarkGval/accessors_method_parsing-4                         1000000              1729 ns/op
BenchmarkGval/accessors_method_parameter_evaluation-4            1000000              2162 ns/op
BenchmarkGval/accessors_method_parameter_parsing-4                500000              2618 ns/op
BenchmarkGval/nested_accessors_evaluation-4                      2000000               681 ns/op
BenchmarkGval/nested_accessors_parsing-4                         1000000              2115 ns/op
BenchmarkRandom-4                                                 500000              3631 ns/op
ok

API Breaks

Gval is designed with easy expandability in mind and API breaks will be avoided if possible. If API breaks are unavoidable they wil be explicitly stated via an increased major version number.


Credits to Reene French for the gophers.

Comments
  • Date evaluation

    Date evaluation

    Hello,

    First your library looks great. Thank you for sharing. I would like to evaluate date + duration: date(myObject.date) + 3d I was wondering what was the best approach: adding a new language? Cheers, JC

  • Without space after && leading error

    Without space after && leading error

    package main
    
    import (
    	"fmt"
    	"github.com/PaesslerAG/gval"
    )
    
    func main() {
    	vars := map[string]interface{}{"name": true}
    
    	value, err := gval.Evaluate("true&&name", vars)
    	if err != nil {
    		fmt.Println(err)
    	}
    	fmt.Println(value)
    
    	value, err = gval.Evaluate("true&& name", vars)
    	if err != nil {
    		fmt.Println(err)
    	}
    
    	fmt.Println(value)
    }
    

    runing result:

    parsing error: true&&name	 - 1:8 unknown operator &&n
    <nil>
    true
    

    My expected result is the first expression also working fine.

  • Error callback instead of terminating error in JSON

    Error callback instead of terminating error in JSON

    Hi and thanks for this greate piece of software! :)

    We're using it to do sensor mapping:

    • i.e fetch sensor data as a single HTTP request
    • map those sensors into our own sensor model.

    This works great, except when for some reason we do not get one or more needed sensor data points. In this case it will error out.

    Could it be possible to do a callback to an error function instead, and if that error function do return an error, it will fail, otherwise it will remove the json key or set the value to nil (or a value returned by the error function?

    func OnError(c context.Context, key, expr string, e error) (val interface{], err error) {}
    

    Example mapping document that we use (partial)

    { "tz": TZ,
    	"sensors": { 
    			"IDT_O3": float64(datapoint["1!6WHH6DKH3CCVW-"].Value),
    			"IDT_O5": float64(datapoint["1!6WHH6DKHADPZ1N"].Value), 
                             "AEUPS": float64(datapoint["1!6WHH6DKHICVDSS"].Value * 1000) }
    }
    

    The error function may act as a OnErrorResumeNext or a circuit-breaker (the latter is how it currently works).

    What do you think about this? Is is hard to implement in your library, would it fit?

    To maybe do other than current error handling when this or this returns an error?

    p.Camouflage("object", ',', '}')
    key, err := p.ParseExpression(c)
    if err != nil {
      return nil, err
    }
    

    Cheers, Mario :)

  • unable to get value

    unable to get value

    i have run the following with jsonPath

    package main
    
    func main() {
    	v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    
    	value, err := jsonpath.Get("$.*.*.val", v)
    	if err != nil {
    		fmt.Println(err)
    		os.Exit(1)
    	}
    
    	for _, value := range value.([]interface{}) {
    		fmt.Println(value)
    	}
    }
    
    

    and this is my output $.*.*.valmap[foo:map[bar:map[val:3]]]3

    however if i run

    v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    value, err := gval.Evaluate("$.*.*.val", v, jsonpath.Language())
    

    it returns

    [] am i using this wrong?

    i would expect it to return

    3 and if i run

    v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    value, err := gval.Evaluate("$.*.*.val == 3", v, jsonpath.Language())
    

    i would expect

    true 
    
  • Case Mismatch: Able to create Evaluable but actual evaluate fails

    Case Mismatch: Able to create Evaluable but actual evaluate fails

    Hi,

    I am using gval to build an excel formula parser. Repo name is efp. Check efp_test.go. You can change case in expression to see behavior.

    In excel, all function names are capitalized so, I have created Functions with Capital names. Language allows creation of non-null Evaluable even when case does not match. But, when Eval* function is called on Evaluable the results are incorrect.

    The function mapped to keyword is called correctly and the value returned from the function is accurate but, it seems this value is lost after evaluation. Any thoughts?

    Ideally Parse should fail in this scenario so that users get feedback when expression is built improperly.

  • Guidance; abusing gval to set keys

    Guidance; abusing gval to set keys

    Hey folks; first up thanks for gval... it's a seriously awesome little library. Hard to believe it doesn't have more adoption!

    To the point... I'm using gval in a little tool I wrote to smash up kubernetes manifests. https://github.com/mikesimons/kpatch. Please excuse the nasty code; it's prototype grade at the moment.

    I've implemented an = infix operator and it works pretty well to overwrite values that already exist.

    The problem is that because I can't get the verbatim key from the left expression I'm having to set values based on a walk of the input structure and matching on reflection values. It works but it means that if the key doesn't exist no value matches and the value is not set.

    To get around it I implemented a set function but from a user experience perspective I'd really like to be able to do some.key = "value" even if some.key wasn't previously set.

    I had a go with an InfixEvalExpression but due to the input being wrapped in an Evaluable func I couldn't get the value out.

    Any advice on a way to do this (even if it means patching gval)?

  • Undefined attributes key returns error

    Undefined attributes key returns error

    Hi, I'm trying to create a conditional expression if a variable is nil, but it throws an Error instead.

    parameters := make(map[string]interface)
    parameters["foo"] = "bar"
    
    expression := "fooz ?? foo"
    result, err := gval.Evaluate(expression, parameters)
    if err != nil {
      panic(err)
    }
    

    Throw error unknown parameter fooz when it should be bar. I've tried to solve this problem by change return error value to a nil value from private function func variable(path ...Evaluable)

  • Extending variable() to support custom selectors on value

    Extending variable() to support custom selectors on value

    I'm struggling to get a bit more control over how path selection is done.

    I would like to expose one of my structs with prop that's internally slice of structs ({ name, value, position }) to be accessed in more "fluent" way (record.values.foo or record.values.bar[0]). More info here: https://github.com/cortezaproject/corteza-server/blob/develop/compose/types/record.go#L38

    Currently, I'm doing this by casting into map[string]interface{} but I would like to avoid this because it requires keeping copies of the original data and constantly syncing and casting.

    I've tried with adding an operator but I have to admit I'm lacking know-how of the internal workings of gval to make even a proof of concept. A custom function (getValue(record, name, position)) would work but that ruins the fluentness :)

    After digging more into gval, I'm thniking that a) variable fn could be modified to get additional case that would cover a special interface interface { GvalSelect(key string) (interface{}, error) } (I'm not 100% happy with the fn name here... ).

    If you think this would be a good approach I can open a PR in next couple of days.

  • Make ident and parentheses parsers available separately from Base()

    Make ident and parentheses parsers available separately from Base()

    When implementing a language with custom operators, only a part of the Base() is useful, yet you can't easily get that part without the rest of the features.

    An example of these parts are private parse*() functions. While things like parseString() can be easily re-implemented, the parseParentheses() and parseIdent() are somewhat more complicated, yet they're not available for re-using externally.

    This change keeps the Base() as is, but allows one to re-use at least the complex parts of it.

  • Publish new version for DecimalArithmetic

    Publish new version for DecimalArithmetic

    Hi Guys,

    I have a requirement for the recently added DecimalArithmetic functionality added here by @machship-mm.

    However, this change is not yet published in the latest release. @generikvault Would it be possible for you or someone else to release a new version?

    Thanks!

  • Handle decimal/money arithmetic

    Handle decimal/money arithmetic

    This library is fantastic!

    The one issue I've found is that the Arithmetic is done as floating point numbers (which is fine for that use case) but this falls short when doing math involving decimals.

    An example:

    func TestDecimalArithmetic(t *testing.T) {
    	input := ArithmeticVariables{
    		NumberMap: map[string]float64{
    			"x": 12.5,
    			"y": -5,
    		},
    		Expression: "(x * 12.146) - y",
    	}
    
    	result, _ := gval.Arithmetic().Evaluate(input.Expression, input.NumberMap)
    
    	want := 156.825
    	if result != want {
    		t.Errorf("wanted %f; got %f", want, result)
    		t.FailNow()
    	}
    
    	t.Log(result)
    }
    

    This results in this output:

    === RUN   TestDecimalArithmetic
        arithmetic_test.go:46: wanted 156.825000; got 156.825000
    --- FAIL: TestDecimalArithmetic (0.00s)
    

    When inspecting the variable values, I can see that result has the actual value of 156.82500000000002.

    Do you have any suggestions for what I should do, or is this even a solvable problem?

    I was thinking that there could be an EvaluateDecimal method, which instead of treating numbers as floats, would treat them as a decimal type? For decimals, I use github.com/shopspring/decimal.

  • fix negation in decimal language

    fix negation in decimal language

    Happened to notice this by inspection.

    I was even getting some weird errors with just a plain "-1", since I suspect this gets parsed as a negate on the literal "1", but I didn't check too deep into it. But my initial concern was about precision, so I've added a test to that effect as well.

  • Evaluating strings with backslashes raises parsing errors

    Evaluating strings with backslashes raises parsing errors

    Env

    This is on github.com/PaesslerAG/gval v1.2.1

    Issue

    We have a custom handler for regular expressions (match(value, regex)) where the regex can be configured from the front-end (web application).

    For example, inputting match(variable, "[a-z]\d") would be presented as (when dumping the value) "match(variable, \"[a-z]\\d\")" (which seems correct to me), but when passed to gval, it raises parsing error: match(variable, "[a-z]\d") :1:11 - 1:20 could not parse string: invalid syntax.

    If I bypass all of the surrounding code and run it directly as so x, err := gval.Evaluate("match(variable, \"[a-z]\\d\")", map[string]any{"a": "b"}) the error is the same.

    Another example would be running this x, err := gval.Evaluate("\"\\\"", map[string]any{"a": "b"}) -- same error in regards to the invalid string (parsing error: "\" :1:1 - 1:4 could not parse string: invalid syntax).

    But then, if I do this x, err := gval.Evaluate("\"apple\\banana\"", map[string]any{"a": "b"}), it passes and outputs (when running spew.Dump(x, err))

    (string) (len=11) "apple\banana"
    (interface {}) <nil>
    

    Conclusion

    Am I doing something wrong with my strings, or is this a (un)intentional edge case?

    If this needs more investigation I can take a look through the code and propose a fix as well, but I'd appreciate some notes on where would be a good place to start/your suspicions about what's wrong

  • Add support for typed map and slices with method calls

    Add support for typed map and slices with method calls

    I noticed that gval didn't support expressions for typed maps and slice method call.

    Example map:

    type MyMap map[string][]int
    
    func (m MyMap) Sum(key string) int {
        values, ok := m[key]
        if !ok {
            return -1
        }
        sum := 0
        for _, v := range values {
            sum += v
        }
        return sum
    }
    
  • Feature Request: high order functions (collection)

    Feature Request: high order functions (collection)

    What is it?

    Some special functions like filter, map, exists, all.

    For example:

    ["project1/text.txt", "project2/text.txt"].exists(t => t.startsWith("project1"))
    ; => true
    

    Why?

    This would simplify a lot of some operations that would require very specific custom functions.


    Let me know if that's something you have thought about it.

  • Allow dashes in identfiers

    Allow dashes in identfiers

    I have the case that one of my identifiers contains a -, like type-id. The parser fails because - is not an allowed rune in an identifier.

    Example:

    func main() {
        vars := map[string]interface{}{"type-id": "4"}
        expr := `type-id == "4"`
        e, err := gval.Full().NewEvaluable(expr)
        if err != nil {
            return
        }
        value, err := e.EvalBool(context.Background(), vars)
        if err != nil {
            return
        }
        fmt.Println(value)
    }
    

    I could open a PR, but the fix is so simple: Add the condition (pos > 0 && r == '-') to https://github.com/PaesslerAG/gval/blob/master/parser.go#L33 . The example works, I tested it even for multiple dashes. But I'm new to gval, so I might miss some cases.

Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Jan 1, 2023
Scriptable interpreter written in golang
Scriptable interpreter written in golang

Anko Anko is a scriptable interpreter written in Go. (Picture licensed under CC BY-SA 3.0, photo by Ocdp) Usage Example - Embedded package main impor

Jan 1, 2023
Gentee - script programming language for automation. It uses VM and compiler written in Go (Golang).

Gentee script programming language Gentee is a free open source script programming language. The Gentee programming language is designed to create scr

Dec 15, 2022
PHP bindings for the Go programming language (Golang)

PHP bindings for Go This package implements support for executing PHP scripts, exporting Go variables for use in PHP contexts, attaching Go method rec

Jan 1, 2023
Expression evaluation in golang
Expression evaluation in golang

Gval Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions. Evaluate Gval can evaluate expressio

Dec 27, 2022
Arbitrary expression evaluation for golang

govaluate Provides support for evaluating arbitrary C-like artithmetic/string expressions. Why can't you just write these expressions in code? Sometim

Jan 2, 2023
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Jan 3, 2023
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Jan 1, 2023
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Dec 30, 2022
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Dec 24, 2022
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Dec 30, 2022
Simple expression evaluation engine for Go

??️ chili Currently in development, Unstable (API may change in future) Simple expression evaluation engine. Expression is one liner that evalutes int

Nov 8, 2022
A Golang program for a colleague to help in calculating the ratio between the points obtained in a test and the corresponding evaluation in tenths.
A Golang program for a colleague to help in calculating the ratio between the points obtained in a test and the corresponding evaluation in tenths.

A Golang program for a colleague to help in calculating the ratio between the points obtained in a test and the corresponding evaluation in tenths. If you have not the compiled file (.exe) you can build it with the Go compiler.

Jul 7, 2022
Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert
Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert

DeepAlert DeepAlert is a serverless framework for automatic response of security alert. Overview DeepAlert receives a security alert that is event of

Jan 3, 2023
Netpoltool - CLI evaluation of Kubernetes NetworkPolicys with detailed output to aid debugging.

netpoltool CLI evaluation of Kubernetes NetworkPolicys with detailed output helpful for debugging. Given source and destination pods, identify the Net

Jan 8, 2022
Singlestore event analytics - Evaluation of sortable ID generation schemes

Singlestore event analytics - Evaluation of sortable ID generation schemes

Jan 25, 2022
Implements a simple floating point arithmetic expression evaluator in Go (golang).

evaler https://github.com/soniah/evaler Package evaler implements a simple floating point arithmetic expression evaluator. Evaler uses Dijkstra's Shun

Sep 27, 2022
JSON query expression library in Golang.

jsonql JSON query expression library in Golang. This library enables query against JSON. Currently supported operators are: (precedences from low to h

Dec 31, 2022
Implements a simple floating point arithmetic expression evaluator in Go (golang).

evaler https://github.com/soniah/evaler Package evaler implements a simple floating point arithmetic expression evaluator. Evaler uses Dijkstra's Shun

Sep 27, 2022
An extremely fast Go (golang) HTTP router that supports regular expression route matching. Comes with full support for building RESTful APIs.

ozzo-routing You may consider using go-rest-api to jumpstart your new RESTful applications with ozzo-routing. Description ozzo-routing is a Go package

Dec 31, 2022