I recently (re-)discovered the OPA project. This issue is about deprecating the ORY Ladon engine and aligning ORY Keto with OPA. The decision is not yet made and we are looking for valuable input regarding this.

OPA allows you to write authorization logic using a language specifically designed for that, called rego. Syntax is very go-like. Due to this, OPA is capable of providing all sorts of authorization mechanisms, like RBAC, ABAC, ACL, AWS IAM Policies, and more. In fact, I believe that ORY Ladon's logic is implementable using rego. I'm not sure if that holds true for conditions which still needs verification from my side.

Let's take a look at the current downsides of each project.

I believe that policy documents as implemented by ORY Ladon are very powerful, but also very complicated. Many developers struggle with proper resource & action naming. I think that regular expressions have their place here, but many developers struggle with writing and testing regular expressions and variable substitution is very flaky (currently only used in the ORY Oathkeeper adapter for ORY Keto iirc) from a ux perspective. Also, regular expressions do not scale well, especially if read from the database. I think we can fix this with caching, but that is not fixing the problem itself, only the symptom.

On the same hand, OPA is limited. I think rego is great for developers that really want to jump into this (like me). But it's a new syntax, new language, and new tools. I think the language is not incredibly intuitive and not always readable:

I believe that policy documents are, in general, quite complicated. In my opinion, rego has a steep learning curve as well. Can you tell me immediately what this does?

sod_roles = [
    ["create-payment", "approve-payment"],
    ["create-vendor", "pay-vendor"]
]

sod_violation[user] {
    role1 = user_role[user][_]
    role2 = user_role[user][_]
    sod_roles[_] = [role1, role2]
}

At least I can not - the point I'm trying to make is, you have to learn this.

OPA comes with a REST API but it's really a parser and execution engine. It parses rego files and executes the logic based on data you provide. The result is always true or false, depending on the authorization result.

The server is limited. It stores everything in-memory so pushing logic to the server is not realistic with more than one server running. Instead, you'll probably have to write a CI pipeline which builds a docker image that has all your policy definitions. IMO that can be very nice, especially if you have rego test as part of that pipeline. But, from my experience, most people do not want or know how to do these things.

Coming back to policies for a moment: Most developers do not need AWS IAM Policies, simple role management is enough. By the way, the Google Cloud Platform migrated completely to RBAC/ACL as well recently (at least in the UI) - very few people want to deal with complicated JSON documents. And many make mistakes, which is evident due to the many S3 leaks we're seeing recently (caused by misconfigured buckets, well really misconfigured AWS IAM Policies). I think the same would have happened if AWS used rego, or it would have probably caused less people to use this feature at all.

My vision for ORY Keto is to have a "policy decision point" (someone that says if something is allowed or not) that just works. I also believe that it should be possible to use several well-known patterns out of the box, this includes RBAC, ACL, ABAC and ORY Ladon Policies (for BC compatibility) and well, maybe your own rego definitions? I will experiment this month with different concepts and try to migrate Ladon Policies on top of rego. My preliminary tests showed that we can get a 10x performance improvement for simple use cases. We'll see how well this does for advanced ones.

We write this software is for you, so please participate in the discussion and leave your ideas and comments below.

Describe the bug Keto's latency increases with policies and resources

To Reproduce Steps to reproduce the behavior:

1. (Optionally) Start with a fresh database
2. Create 30k regex policies with 1 or more resources
3. Perform allowed checks
4. Witness requests take 1-10 seconds to complete

Expected behavior Access allowed checks under 100ms

Version:

• Environment: MacOS Mojave, Postgres 11
• Version: Master (0.3.1)

Additional context Initially, I noticed Keto requests taking longer than 1 minute in pods running on Kubernetes when our policy count exceeded 30k. We also had memory issues but it may be unrelated. I was unable to run a memory profile using ppof on our images, the pod would crash OOM before the memory profile was written.

The policy causing the latency had 16 resources. They all had 2 wildcards. Ex: tenant:<.*>:resource:foo<.*>

Observations about latency:

• The number of resources directly affects latency.
• The number of policies (unused or not) affects latency
• SERVE_MAX_AGE caused 500 errors with this many policies

A temporary solution to this would be to split my policy into 16 individual policies, but 1-2 second requests (running locally) still seems extreme.

Describe the bug Version [0.2.2-sandbox+oryOS.10] keto serve => Unable to initialize compiler: lstat /go: no such file or directory

Version [v0.1.9-sandbox+oryOS.9] keto serve => work well

To Reproduce Steps to reproduce the behavior:

1. Run keto serve
2. See error: Unable to initialize compiler: lstat /go: no such file or directory

Version:

• Environment: [binaries]
• Version [0.2.2-sandbox+oryOS.10]

Is your feature request related to a problem? Please describe.

We need to define all SQL queries required for fetching / querying relations.

Describe the solution you'd like

Discuss SQL queries in this issue.

Is your feature request related to a problem? Please describe.

Currently we use protoc wich is quite limited. A better tool might be https://github.com/uber/prototool It comes with linting. Also interesting: https://github.com/googleapis/gapic-generator

We might want to use a gateway to avoid writing REST APIs.

Related issue #66

Proposed changes

Adds support for the "glob" flavor, following the same patterns as "exact" and "regex".

Checklist

• [x] I have read the contributing guidelines
• [x] I confirm that this pull request does not address a security vulnerability. If this pull request addresses a security vulnerability, I confirm that I got green light (please contact [email protected]) from the maintainers to push the changes.
• [x] I signed the Developer's Certificate of Origin by signing my commit(s). You can amend your signature to the most recent commit by using git commit --amend -s. If you amend the commit, you might need to force push using git push --force HEAD:<branch>. Please be very careful when using force push.
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have added necessary documentation within the code base (if appropriate)
• [x] I have documented my changes in the developer guide (if appropriate) (https://github.com/ory/docs/pull/102)

Further comments

Bug or Unclear Documentation Conditional function based on Context Conditions does not Apply Tested with EqualsSubjectCondition and StringEqualCondition

To Reproduce Steps to reproduce the behavior:

1. Create a exact Policie with the resulting Json Response on: /engines/acp/ory/exact/policies

[{
   "id": "test-users-alice",
   "description": "",
   "subjects": ["users:alice"],
   "resources": ["resources:users:alice"],
   "actions": [ "read", "update"],
   "effect": "allow",
   "conditions": [ {
      "subject": {
         "type": "EqualsSubjectCondition"
      }
   }]
}]

2. Request on /engines/acp/ory/exact/allowed with the Request Body:

{
    "action": "read",
    "subject": "users:alice",
    "resource": "resources:users:alice",
    "context": {
        "subject": "users:alice"
    }
}

3. Results in following Result

{
    "allowed": false
}

Expected behavior Expected is to following Result:

{
    "allowed": true
}

Version:

• Environment: Docker
• Version: oryd/keto:v0.2.2-sandbox_oryOS.10

This PR contains first work towards supporting automatic subject/object encodings into UUIDs, as discussed in #792.

Related issue(s)

#792

Checklist

• [x] I have read the contributing guidelines.
• [x] I have referenced an issue containing the design document if my change introduces a new feature.
• [x] I am following the contributing code guidelines.
• [x] I have read the security policy.
• [x] I confirm that this pull request does not address a security vulnerability. If this pull request addresses a security. vulnerability, I confirm that I got green light (please contact [email protected]) from the maintainers to push the changes.
• [x] I have added tests that prove my fix is effective or that my feature works.
• [ ] I have added or changed the documentation.

Future work

In order to complete the work for #792, the following items still need to be adressed (in this or future PRs):

• [ ] RFC: Should the new encoding be enabled unconditionally, or opt-in as suggested in https://github.com/ory/keto/issues/792#issue-1071582579?
• [ ] Use the new mapping methods in the API when reading or writing relation tuples.

Currently the column definitions use an arbitrary fixed string length. Researching a bit revealed that there are basically no advantages with that with any of our supported database systems. We should just make those strings variable length to allow applications to use any format they like. Making clear that using huge strings and pointing to the databases' documentation should be enough IMO.

Describe the bug

When importing Keto we need to do the same replacements that are currently in master branch and in version v0.6.0.

However, the replacement for kratos-client-go is on version v0.5.4-alpha.1.0.20210210170256-960b093d8bf9 which is not compatible with latest Kratos version.

When removing this replacement the error

go: github.com/ory/[email protected] requires
        github.com/ory/[email protected] requires
        github.com/ory/[email protected] requires
        github.com/ory/[email protected] requires
        github.com/ory/[email protected] requires
        github.com/ory/[email protected]: invalid version: unknown revision 000000000000
        

occurs. As long as the replacement is there, it is obviously not possible to update kratos-client-go.

Reproducing the bug

Just have a Go project importing github.com/ory/keto v0.6.0-alpha.3, use the replacements (otherwise it's not working anyway) and then try to use kratos-client-go on its latest version.

Expected behavior

It should be possible to use latest Kratos (0.6.x) and Keto (0.6.x) version together.

Wildcards in the members of groups are not interpreted as wildcards for flavor glob.

Applies to recent versions of keto, e.g., to commit db94481.

The reason seems to be at the end of the file engine/ladon/rego/glob/main.rego in the function match_subjects. The intended implementation is preceded by the implementation for the flavor exact.

Add something like

decide_allow([group_policy], [{"id": "groups:6", "members": ["group-*"]}]) with input as {"resource": "articles:6", "subject": "group-member", "action": "actions:6"}

to main_test.rego for a corresponding test.

Bumps github.com/ory/x from 0.0.502 to 0.0.524.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

This patch removes query components which cause CockroachDB to perform a full-range query across all leaseholders for LIMIT 1 queries. Essentially, CockroachDB will query all potential nodes which might hold data when ORDER BY is used, or when the LIMIT indicates that there are rows missing.

This can drop the query latency to a few ms versus hundreds of ms on a CockroachDB REGIONAL BY ROW table, without impact on the business logic.

Closes #1167

Further Comments

I wasn't quite sure how to write tests for this. One way would be to test the resulting SQL query, but we do not have such capabilities yet.

Preflight checklist

• [X] I could not find a solution in the existing issues, docs, nor discussions.
• [X] I agree to follow this project's Code of Conduct.
• [X] I have read and am following this repository's Contribution Guidelines.
• [X] This issue affects my Ory Network project.
• [ ] I have joined the Ory Community Slack.
• [ ] I am signed up to the Ory Security Patch Newsletter.

Describe the bug

Calls to checkDirect perform a GetRelationTuples query with x.WithSize(1) to check whether a tuple was found or not.

https://github.com/ory/keto/blob/a39325ce6ad9a401731cfcf5961932232f92145e/internal/check/engine.go#L162-L179

For some reason though, we use the limit + 1 in the query:

https://github.com/ory/keto/blob/a39325ce6ad9a401731cfcf5961932232f92145e/internal/persistence/sql/relationtuples.go#L215-L218

So what should be LIMIT 1 (we have a binary question - does it exist or not?) we end up with LIMIT 2

Reproducing the bug

Look at a GetActiveProject trace in Grafana.

Relevant log output

No response

Relevant configuration

No response

Version

master

On which operating system are you observing this issue?

None

In which environment are you deploying?

None

Additional Context

No response

Bumps @grpc/grpc-js from 1.7.3 to 1.8.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps prettier from 2.7.1 to 2.8.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.