Feature request checklist

• [X] There are no issues that match the desired change
• [ ] The change is large enough it can't be addressed with a simple Pull Request
• [ ] If this is a bug, please file a Bug Report.

Change

Currently, cel.Container can be used to specify a namespace for symbol resolution. For example:

env, _ := cel.NewEnv(
	cel.Container("a"),
	cel.Types(&apb.A{}),
)
ast, _ := env.Compile("A{}")
prg, _ := env.Program(ast)
prg.Eval(map[string]interface{}{})

In order to write programs that reference multiple protobuf packages, we are forced to use fully qualified names for types outside the container:

env, _ := cel.NewEnv(
	cel.Container("a"),
	cel.Types(&apb.A{}, &bpb.B{}),
)
ast, _ := env.Compile("b.B{}")
prg, _ := env.Program(ast)
prg.Eval(map[string]interface{}{})

Example

We should provide the ability to resolve multiple namespaces. There are two options:

1. Container could take multiple package names:

func Container(pkgs ...string) EnvOption

2. Env could be initialized with a custom packages.Packager:

func Package(pkg packages.Packager) EnvOption

Alternatives considered

It’s possible that both these methods would be useful. Here is a sample implementation:

package cel

func Container(pkgs ...string) EnvOption {
	return func(e *Env) (*Env, error) {
		return e.Package(packages.NewPackage(pkgs...))
	}
}

func Package(pkg packages.Packager) EnvOption {
	return func(e *Env) (*Env, error) {
		e.pkg = pkg
		return e, nil
	}
}

//////

package packages

type defaultPackage struct {
	pkgs []string
}

func NewPackage(pkgs ...string) packages.Packager {
	seen := make(map[string]bool, len(pkgs))

	var p defaultPackage
	for _, pkg := range pkgs {
		if seen[pkg] {
			continue
		}
		p.pkgs = append(p.pkgs, p.pkgs)
		seen[pkg] = true
	}
	return &p
}

func (p *defaultPackage) Package() string {
	return strings.Join(p.pkgs, " ")
}

func (p *defaultPackage) ResolveCandidateNames(name string) []string {
	if strings.HasPrefix(name, ".") {
		return []string{name[1:]}
	}

	if len(p.pkg) == 0 {
		return []string{name}
	}

	for _, nextPkg := range p.pkgs {
		candidates := []string{nextPkg + "." + name}
		for i := strings.LastIndex(nextPkg, "."); i >= 0; i = strings.LastIndex(nextPkg, ".") {
			nextPkg = nextPkg[:i]
			candidates = append(candidates, nextPkg+"."+name)
		}
	}

	return append(candidates, name)
}

This code is based on FR #306 which requested support for built-in substrings. While substrings are not part of the core CEL spec, such functions are a common ask and the following is an implementation of such functions as extensions which can be layered onto the core CEL spec.

Closes #306

This PR fixes instances where cost estimation is used on expressions where a CallExpr is nested inside of another CallExpr and math.MaxUint64 is incorrectly returned. The test case included with this PR shows an example of where this can happen; == is treated as a function call/CallExpr, and will call sizeEstimate on its left and right-hand expressions. sizeEstimate will then try to call AstNode.ComputedSize before trying EstimateSize and finally returning math.MaxUint64 for an upper limit.

That constant will get returned for operator-based CallExprs inside CallExprs whenever the inner one lacks an EstimateSize, such as with list.size(). Instead, this PR checks for CallExpr inside SizeEstimate and returns a min/max cost of 1.

The original issue was first reported in https://github.com/kubernetes/kubernetes/issues/111769.

Unless I am missing something, it is currently not possible to build an Ast from an exprpb.ParsedExpr or exprpb.CheckedExpr with source contents. As a consequence, if an Ast built from ParsedExprToAst or CheckedExprToAst is checked or executed, the error messages will not show code snippets.

Introduce a CheckedExprToAstWithSource and ParsedExprToAstWithSource to allow providing the source contents.

Unary type-conversion functions with literal arguments should be evaluated eagerly when the cel.EvalOptions(cel.OptOptimize) flag is set. The conversion will also ensure that if an error is encountered when evaluating these functions, such as when constructing timestamps or durations from strings, the evaluation will result in an error in the env.Program call.

This is not a perfect fix for #359, but it can be used to assist with correctness checks.

Hi, sorry for the lame question -- how can I compare timestamps using CEL + this library?

I tried timestamp(0) == timestamp(0) but I get false every time.

Is there a reference I could read about how to do this?

We're evaluating this for our filter expressions and we found a weird behaviour which I don't know if its related to our environment or the lib itself. The issue comes when using the in operator and defining a list with different types. As far as i know, the parsed expression checking enforces this by default, or this is what i understood by reading this paragraph on the langdef spec:

The type checker also introduces the dyn type, which is the union of all other types. Therefore the type checker could accept a list of heterogeneous values as dyn([1, 3.14, "foo"]), which is tiven the type list(dyn). The standard function dyn has no effect at runtime, but signals to the type checker that its argument should be considered of type dyn, list(dyn), or a dyn-valued map.

Take this as an example:

• Declare Parser

// Parser is our expr parser
type Parser struct {
	env *checker.Env
}

// NewParser instantiates the Parser object
func NewParser(message proto.Message, allowedFields []string) (*Parser, error) {
	// attach protobuf & initialize env
	pkg := proto.MessageName(message)
	typeProvider := types.NewProvider(message)
	exprType := decls.NewObjectType(pkg)

	// create standard env
	env := checker.NewEnv(packages.NewPackage(pkg), typeProvider)
	if err := env.Add(standardDeclarations()...); err != nil {
		return nil, err
	}

	// add types tailored to our specific use-case.
	for _, filterable := range allowedFields {
		fieldType, ok := typeProvider.FindFieldType(exprType, filterable)
		if !ok {
			return nil, fmt.Errorf("expr: couldn't find field %s", filterable)
		}
		err := env.Add(decls.NewIdent(filterable, fieldType.Type, nil))
		if err != nil {
			return nil, err
		}
	}

	return &Parser{env: env}, nil
}

func standardDeclarations() []*exprpb.Decl {
	return []*exprpb.Decl{
		decls.NewFunction(operators.LogicalAnd,
			decls.NewOverload(overloads.LogicalAnd,
				[]*exprpb.Type{decls.Bool, decls.Bool}, decls.Bool)),
		decls.NewFunction(operators.LogicalOr,
			decls.NewOverload(overloads.LogicalOr,
				[]*exprpb.Type{decls.Bool, decls.Bool}, decls.Bool)),
		decls.NewFunction(operators.Equals,
			decls.NewOverload(overloads.Equals,
				[]*exprpb.Type{decls.String, decls.String}, decls.Bool)),
		decls.NewFunction(operators.In,
			decls.NewOverload(overloads.InList,
				[]*exprpb.Type{decls.String, decls.NewListType(decls.String)}, decls.Bool)),
	}
}

• Parse filter

// Parse produces a database friendly expr from a cel string expr
func (e *Parser) Parse(filter string) (*Expr, error) {
	if filter == "" {
		return nil, nil
	}
	// parse expr
	src := common.NewTextSource(filter)
	expr, errs := parser.Parse(src)
	if len(errs.GetErrors()) != 0 {
		return nil, errors.New(errs.ToDisplayString())
	}

	// check expr against env
	checkedExpr, errs := checker.Check(expr, src, e.env)
	if len(errs.GetErrors()) != 0 {
		return nil, errors.New(errs.ToDisplayString())
	}

	return &Expr{expr: checkedExpr.Expr}, nil
}

I would expect the checker to fail but it returns the checked expression without complaining.

Can anyone point to the right direction if i'm missing something?

Thanks for this great library by the way.

Introduce runtime cost calculation into CEL.

The work is part of this issue

The current PR includes:

• Add runtime cost calculation into evaluation path
• Test to verify runtime cost calculation
• Test to verify runtime cost fell into estimation cost range
• Test to allow cost calculation for extension function

Todo in following PR:

• Add cancellation
• Add more test for large input
• Add test coverage for aligning runtime cost with estimate cost

Hi,

Awesome project 😀 But I found this in README.md:

... https://github.com/antonmedv/expr ... StartsWith(name, Concat("/groups/", group)) // Expr BenchmarkExpr-8 1000000 1402 ns/op

Please, update README.md with new results:

❯❯❯ go test -bench=startswith
goos: darwin
goarch: amd64
pkg: github.com/antonmedv/golang-expression-evaluation-comparison
Benchmark_celgo_startswith-8   	 3000000	       466 ns/op
Benchmark_expr_startswith-8    	 5000000	       362 ns/op
PASS
ok  	github.com/antonmedv/golang-expression-evaluation-comparison	4.081s

Describe the bug As mentioned in https://github.com/google/cel-go#install

go mod init <my-cel-app>
go build ./...

or

go get -u github.com/google/cel-go/...

Does not work and results in the following error:

go build -o myproj.bin -v
go: finding github.com/google/cel-go/interpreter/functions latest
go: finding github.com/google/cel-go/common/types latest
go: finding github.com/google/cel-go/cel latest
go: finding github.com/google/cel-go/common/types/ref latest
go: finding github.com/google/cel-go/checker/decls latest
go: finding github.com/google/cel-go/interpreter latest
go: finding github.com/google/cel-go/common latest
Fetching https://github.com?go-get=1
go: finding github.com/google/cel-go/checker latest
Parsing meta tags from https://github.com?go-get=1 (status code 200)
build company.tech/rhnvrm/myproj cannot load github.com/google/cel-go/cel: cannot find module providing package github.com/google/cel-go/cel

go get -u github.com/google/cel-go/...
...
go: github.com/golang/[email protected]: parsing go.mod: unexpected module path "golang.org/x/lint"

To fix this for now, I have done a git clone and added a replace directive

This is my go.mod

module company.tech/rhnvrm/myproj

go 1.12

require (
	github.com/golang/protobuf v1.3.0
	github.com/google/cel-go v0.2.0
	google.golang.org/genproto v0.0.0-20190227213309-4f5b463f9597
)

replace github.com/google/cel-go v0.2.0 => ./celgo

It seems that the generic text proto CompactStringMarshal is more expensive that hand-made one. Looking for feedback. I think we probably want to use a Struct as a hashmap key, but that would require building a new family of structs mirroring Types, since using proto Type does not seem to work.

Describe the bug

There doesn't appear to be a syntax for checking the existence of a key in a dictionary if the key contains characters that are disallowed in field names. For example "foo/bar".

I have reviewed the documentation from the language spec and it seems you're supposed to be able to invoke via:

has(foo.x)

to check the existence of x on the foo object. I infer from that example that a user would use the standard access pattern and that the interpreter would handle the conversion for you. That is to say that if foo.x is a map[string]string you would like to check for key existence, you might try:

has(foo.x["bar"])

To Reproduce Check which components this affects:

I think it's the interpreter, but I'm not entirely sure. Sorry!

• [ ] parser
• [ ] checker
• [X] interpreter

Sample expression and input that reproduces the issue:

// sample expression string

Test setup:

Add a test case to interpreter/interpreter_test.go:

    { // N.B. this one passes already, so the setup is sane-ish
      name:      "has_able_to_detect_maps_with_complex_keys_success",
      container: "google.expr.proto3.test",
      types:     []proto.Message{&proto3pb.TestAllTypes{}},
      expr: `has(TestAllTypes{
        map_string_string: {
          'hello': 'world',
        },
      }.map_string_string.hello)`,
      out: types.True,
    },
    {
/* This testcase fails with: 
    interpreter_test.go:1579: has_able_to_detect_maps_with_complex_keys_attempt_1: ERROR: <input>:1:4: invalid argument to has() macro
         | has(TestAllTypes{
         | ...^
*/
      name:      "has_able_to_detect_maps_with_complex_keys_attempt_1",
      container: "google.expr.proto3.test",
      types:     []proto.Message{&proto3pb.TestAllTypes{}},
      expr: `has(TestAllTypes{
        map_string_string: {
          'hello/world': 'moo',
        },
      }.map_string_string["hello/world"])`,
    },
    {
/* This testcase fails with: 
    interpreter_test.go:1585: has_able_to_detect_maps_with_complex_keys_attempt_2: ERROR: <input>:1:4: invalid argument to has() macro
         | has(TestAllTypes{
         | ...^
*/
      name:      "has_able_to_detect_maps_with_complex_keys_attempt_2",
      container: "google.expr.proto3.test",
      types:     []proto.Message{&proto3pb.TestAllTypes{}},
      expr: `has(TestAllTypes{
        map_string_string: {
          'hello/world': 'moo',
        },
      }.map_string_string.hello/world)`,
    },

Expected behavior

A function for checking the existence of a map key.

Additional context

I think this is possibly solvable by creating my own macro that takes an dict (map[string]any) and a key (in this case string) and returns an any iff the key exists, otherwise returns a default value. Is there a better way to approach this without extending the language with my own macro?

This PR adds a format receiver function to CEL strings. The syntax is similar to Python's str.format; {n} is replaced with the nth argument passed to format, so "this {0} a {1}".format(["is", "test"]) turns into "this is a test".

I have a self-defination struct like this

type student {
    name string
    age int
}

then, I generate a array of "student"

students := make([]*student, 0)
students = append(students, &student{"123",18}),
students = append(students, &student{"456",19}),
students = append(students, &student{"789",20}),

if the age of all the students is more than 17,it returns true,otherwise,returns false What should I do?

Feature request checklist

• [X] There are no issues that match the desired change
• [X] The change is large enough it can't be addressed with a simple Pull Request
• [ ] If this is a bug, please file a Bug Report.

Change Currently, when parse errors are detected, the ANTLR error message is passed through from CEL to the caller with some additional formatting to highlight where the error occurs. A recent update to the CEL grammar resulted in a shift in the error information from ANTLR which has the potential to break upstream tests relying on the exact contents of the original error from ANTLR.

Example Introduce a flag-guarded change to normalize error messages from ANTLR in order to ensure that errors can be kept consistent even in the face of grammar updates.

Alternatives considered Strip all ANTLR error messages and replace them with a general "invalid syntax" message since the error from ANTLR itself is not always easy to comprehend.

Feature request checklist

• [X] There are no issues that match the desired change
• [X] The change is large enough it can't be addressed with a simple Pull Request
• [ ] If this is a bug, please file a Bug Report.

Change The top-level API currently emphasizes using cel.Type values rather than exprpb.Type values in an effort to reduce the protobuf surface area visible on the cel-go APIs. However, the ref.TypeProvider currently exposes a method to FindType and FindFieldType which return an exprpb.Type. This interface is used by a handful of customers to extend the CEL type-system, and creates a disconnect between the top-level API and the extensions.

The change should introduce a new TypeProvider which can wrap the old style and produce cel.Type values from these methods for use within the type-checker and runtimes.

Alternatives considered Provide new methods on the existing TypeProvider interface.

• Functional, but would be a breaking API change for existing users.
• Introduces a more complex migration burden of having to support both the old and new-style methods.

These changes introduce :

• a new interface functions.Overloader for keeping compatiblity with already defined functions.Overload structures
• a new structure functions.ContextOverload for defining context capable overloads
• the extension of the interpreter.Activation interface with context.Context
• a small test for checking that context cancellation can be catched with underlying overloads

The idea was to keep API compatibility while propagating the execution context to overload functions.

Resolves #557