An initial attempt at fixing #13. I am not too familiar with the codebase, so feel free to let me know if this is not the right approach for fixing this issue or if I missed a place where this code needs to be updated 😄

Hi, I was playing around with iamzero (really cool btw, love the work) when I noticed that recommendations wasn't working for me. I was using the iamzero-python-example and the provided guide and it still wasn't working. Then I noticed that the correct information was being transmitted to iamzero from the Python library, but it was erroring on the Go side. Specifically, in pkg/recommendations/arn.go there is a check for regex and it matches with a role but not a user. The error:

... "msg":"web handler error","err":"could not find role in ARN arn:aws:iam::REDACTED:user/USERNAME" ...

I worked on an initial fix for this, but not sure if that's the approach you want to take!

Also reorganises storage objects into a single Storage struct.

This means we use h.Storage.Findings rather than h.findingsStorage and will mean less boilerplate config and injection of storage when we add new metadata entities to be stored.

Adds Postgres as a storage driver to store Findings in. I've also renamed some variables to be a bit clearer. In particular I've renamed Resource to be CloudResourceInstance to indicate a concrete deployed cloud resource.

Temporarily, I've removed some of the CDKResource structs and functionality. We need to shift the CDKResource information away from the current Resource struct and into the CloudResourceTemplate struct once we create it.

I've also added an integration testing package so that we can easily write unit tests that use a real Postgres database, running in a Docker container. These tests are separated from the regular unit tests through a Go build flag as they require Postgres to be available on localhost. These tests run in a new test suite in GitHub actions too.

This PR refactors the project structure to separate the Collector application (responsible for receiving events from IAM Zero clients) and the Console application (the web app where least-privilege permissions are shown). The structure has been inspired by the Jaeger codebase - in particular, I have implemented an all-in-one service which runs both the collector and console together in separate goroutines within the same binary. This should make local development and testing very fast but allow the services to be run separately in a production deployment.

Part of this change separates the ports used for the Collector and Console applications. The Collector has a separate security context to the Console - it is designed to be public facing and only has an endpoint to receive events. The Console on the other hand is intended for authorised users only and allows policy ARNs and data to be read. Separating the ports allows different ingress configurations to be applied for each. In our testing environment we have configured an AWS ALB with SSO to securely access the Console, whereas the Collector is publicly exposed as it currently handles auth at the application level.

Additionally this PR adds support for the Collector to listen to an SQS queue to receive events (we shipped support for the Python library to use customisable transports in https://github.com/common-fate/iamzero-python/pull/6). This functionality is disabled by default and is customisable through CLI flags and environment variables. In future we can use the same pattern to allow new listeners to be added to the Collector.

Breaking changes

• IAM Zero now uses three ports. 13991 is the port that the collector listens for events sent via HTTP on. 14321 is the port that the console application (both the web app and the API) are served on. 10866 is used as an "admin" port for healthchecks (and in future, metrics can be exposed on this port). Using a separate port for healthchecks and metrics allows us to avoid publicly exposing this information in a deployment.

Closes #17.

Our reworked policy editor is great, but the "resource" column in the table is always set to iamzero-test-access-bucket. This is a hold-over from when we did the initial frontend UI implementation, which used fixture data.

A good initial implementation here would be take the "resource" fields from our policy advice - i.e.

						{
							Action: []string{
								"dynamodb:GetShardIterator",
								"dynamodb:Scan",
								"dynamodb:Query",
								"dynamodb:DescribeStream",
								"dynamodb:GetRecords",
								"dynamodb:ListStreams",
							},
							Resource: []string{
								"arn:aws:dynamodb:{{ .Region }}:{{ .Account }}:table/{{ .Table }}/index/*",
								"arn:aws:dynamodb:{{ .Region }}:{{ .Account }}:table/{{ .Table }}/stream/*",
							},
						},

and take the templated variables in the Resource section, excluding the region and the account, as the resource name. So in the above example the resource name would be the DynamoDB table and we could include this in the UI.

Some of the latest changes around making IAM Zero deployable as a service have broken the local workflow.

Loading your iamzero config file (/Users/chrisnorman/.iamzero.ini)
The URL in your config file (https://console.demo.iamzero.dev) was different to the URL your local iamzero server will run on (http://localhost:9090). Updating your config file URL to be http://localhost:9090...
Running local version of iamzero - web console can be accessed at http://localhost:9090
2021/07/16 15:47:41 token storage backend  is not supported
exit status 1

Adds support for tracing with OpenTelemetry. To be flexible with deployment strategies we are using the OpenTelemetry Collector service which allows traces to be forwarded into downstream storage (Jaeger, Zipkin, etc).

This won't work in a deployed environment yet as the collector URL is hardcoded to localhost! But I think if we add tracing earlier rather than later we'll get an idea of performance while we develop IAM Zero. This is not production ready either as currently we sample 100% of traces.

Tracing is only active if IAMZERO_TRACING_ENABLED is set.

This PR adds functionality for users to generate tokens through the IAM Zero console (and through our API!). These tokens are used by IAM Zero clients to authorise against the server while sending IAM events. Closes #20.

Closes #19. The initial implementation relies on handling authentication at the load balancer level (the demonstration I have been testing is an AWS Application Load Balancer with Cognito authentication). In future we should support some additional flexibility, maybe OAuth2.0 auth at the application level.

Currently the IAM Zero console allows authentication via a single token. We should expand this to support multiple users accessing the console under their own identities. An initial implementation could use an external OIDC identity provider such as AWS Cognito (and assume we use an authentication proxy like an application load balancer with Cognito integration)

Bumps minimist from 1.2.5 to 1.2.6.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps async from 2.6.3 to 2.6.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps nanoid from 3.1.20 to 3.3.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

https://github.com/common-fate/iamzero-python-example/pull/6 adds some new examples, including a specific example which calls an AWS API we don't yet provide least-privilege advisories for. Currently when you run this example, this is what IAM Zero shows:

Which is not too helpful to a user!

Even though we don't have an advisory, we still capture a lot of information from the API calls which our users could use to quickly build policies themselves. We should improve the IAM Zero console to allow users to quickly draft policy statements based on our recorded information.

We should group actions together if they are a similar API call - for example, the below policy should only show a single action as the same S3 API has been called multiple times.