🌍 πŸ“‹ A web dashboard to inspect Terraform States

Terraboard

Website: https://terraboard.io

Terraboard Logo

🌍 πŸ“‹ A web dashboard to inspect Terraform States

Docker Pulls Go Report Card Gitter Build Status Coverage Status By Camptocamp

What is it?

Terraboard is a web dashboard to visualize and query Terraform states. It currently features:

  • an overview page listing the most recently updated state files with their activity
  • a state page with state file details, including versions and resource attributes
  • a search interface to query resources by type, name or attributes
  • a diff interface to compare state between versions

It currently supports several remote state backend providers:

Overview

The overview presents all the state files in the S3 bucket, by most recent modification date.

Screenshot Overview

Search

The search view allows to find resources by various criteria.

Screenshot Search

State

The state view presents details of a Terraform state at a given version.

Screenshot State

Compare

From the state view, you can compare the current state version with another version.

Screenshot Compare

Requirements

Independently of the location of your statefiles, Terraboard needs to store an internal version of its dataset. For this purpose it requires a PostgreSQL database. Data resiliency is not paramount though as this dataset can be rebuilt upon your statefiles at anytime.

AWS S3 (state) + DynamoDB (lock)

  • A versioned S3 bucket name with one or more Terraform states, named with a .tfstate suffix
  • AWS credentials with the following IAM permissions over the bucket:
    • s3:GetObject
    • s3:ListBucket
    • s3:ListBucketVersions
    • s3:GetObjectVersion
  • If you want to retrieve lock states from a dynamoDB table, you need to make sure the provided AWS credentials have dynamodb:Scan access to that table.

Terraform Cloud

  • Account on Terraform Cloud
  • Existing organization
  • Token assigned to an organization

Configuration

Terraboard currently supports configuration in three different ways:

  1. Environment variables
  2. CLI parameters
  3. Configuration file (YAML). A configuration file example can be found in the root directory of this repository.

The precedence of configurations is as described below.

Available parameters

CLI ENV YAML Description Default
-V or --version - - Prints version -
-p or --port TERRABOARD_PORT web.port Port to listen on 8080
-c or --config-file CONFIG_FILE - Config File path -
-l or --log-level TERRABOARD_LOG_LEVEL log.level Set log level (debug, info, warn, error, fatal, panic) info
--log-format TERRABOARD_LOG_FORMAT log.format Set log format (plain, json) plain
--db-host DB_HOST db.host Database host db
--db-port DB_PORT db.port Database port 5432
--db-user DB_USER db.user Database user gorm
--db-password DB_PASSWORD db.password Database password -
--db-name DB_NAME db.name Database name gorm
--db-sslmode DB_SSLMODE db.sslmode SSL mode enforced for database access (require, verify-full, verify-ca, disable) require
--no-sync - db.no-sync Do not sync database false
--sync-interval - db.sync-interval DB sync interval (in minutes) 1
--dynamodb-table AWS_DYNAMODB_TABLE aws.dynamodb-table AWS DynamoDB table for locks -
--s3-bucket AWS_BUCKET aws.bucket AWS S3 bucket -
--app-role-arn APP_ROLE_ARN aws.app-role-arn Role ARN to Assume -
--aws-external-id AWS_EXTERNAL_ID aws.external-id External ID to use when assuming role -
--key-prefix AWS_KEY_PREFIX aws.key-prefix AWS Key Prefix -
--file-extension AWS_FILE_EXTENSION aws.file-extension File extension(s) of state files. Use multiple CLI flags or a comma separated list ENV variable .tfstate
--base-url TERRABOARD_BASE_URL web.base-url Base URL /
--logout-url TERRABOARD_LOGOUT_URL web.logout-url Logout URL -
--tfe-address TFE_ADDRESS tfe.tfe-address Terraform Enterprise address for states access -
--tfe-token TFE_TOKEN tfe.tfe-token Terraform Enterprise token for states access -
--tfe-organization TFE_ORGANIZATION tfe.tfe-organization Terraform Enterprise organization for states access -
--gcs-bucket N/A gcp.gcs-buckets Google Cloud Storage buckets to access -
--gcp-sa-key-path GCP_SA_KEY_PATH gcp.gcp-sa-key-path Path to the service account key to use for Google Cloud Storage -
GODEBUG netdns=go DNS resolver to use (see https://github.com/camptocamp/terraboard/issues/118) netdns=cgo

Use with Docker

Docker-compose

Configuration file can be provided to the container using a volume or a configuration.

# Set AWS credentials as environment variables:
export AWS_ACCESS_KEY_ID=<access_key>
export AWS_SECRET_ACCESS_KEY=<access_secret>

# Set AWS configuration as environment variables:
export AWS_DEFAULT_REGION=<AWS default region>
export AWS_BUCKET=<S3 Bucket name>
export AWS_DYNAMODB_TABLE=<Aws DynamoDB Table>

docker-compose up

Then point your browser to http://localhost:8080.

Docker command line

# Set AWS credentials as environment variables:
export AWS_ACCESS_KEY_ID=<access_key>
export AWS_SECRET_ACCESS_KEY=<access_secret>

# Set AWS configuration as environment variables:
export AWS_DEFAULT_REGION=<AWS default region>
export AWS_BUCKET=<S3 Bucket name>
export AWS_DYNAMODB_TABLE=<AWS_DYNAMODB_TABLE>

# Spin up the two containers and a network for them to communciate on:
docker network create terraboard
docker run --name db \
  -e POSTGRES_USER=gorm \
  -e POSTGRES_DB=gorm \
  -e POSTGRES_PASSWORD="" \
  -e GODEBUG="netdns=go" \
  --net terraboard \
  --detach \
  --restart=always \
  postgres:9.5

docker run -p 8080:8080 \
  -e AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \
  -e AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
  -e AWS_REGION="${AWS_DEFAULT_REGION}" \
  -e AWS_BUCKET="${AWS_BUCKET}" \
  -e WS_DYNAMODB_TABLE="${AWS_DYNAMODB_TABLE}" \
  -e DB_PASSWORD="" \
  -e DB_SSLMODE="disable" \
  --net terraboard \
  camptocamp/terraboard:latest

Then point your browser to http://localhost:8080.

Use with Rancher

Camptocamp's Rancher Catalog contains a Terraboard template to automate its installation in Cattle.

Authentication and base URL

Terraboard does not implement authentication. Instead, it is recommended to use an authentication proxy such as oauth2_proxy.

If you need to set a route path for Terraboard, you can set a base URL by passing it as the BASE_URL environment variable.

When using an authentication proxy, Terraboard will retrieve the logged in user and email from the headers passed by the proxy. You can also pass a TERRABOARD_LOGOUT_URL parameter to allow users to sign out of the proxy.

Install from source

$ go get github.com/camptocamp/terraboard

Compatibility Matrix

Terraboard Max Terraform version
0.15.0 0.12.7
0.16.0 0.12.7
0.17.0 0.12.18
0.18.0 0.12.18
0.19.0 0.12.20
0.20.0 0.12.26
0.21.0 0.12.28
0.22.0 0.13.0
1.0.0 0.14.5

Development

Architecture

Terraboard is made of two components:

A server process

The server is written in go and runs a web server which serves:

  • the API on known access points, taking the data from the PostgreSQL database
  • the index page (from static/index.html) on all other URLs

The server also has a routine which regularly (every 1 minute) feeds the PostgreSQL database from the S3 bucket.

A web UI

The UI is an AngularJS application served from index.html. All the UI code can be found in the static/ directory.

Testing

$ docker-compose build && docker-compose up -d
# Point your browser to http://localhost

Contributing

See CONTRIBUTING.md

Owner
Camptocamp
Innovative Solutions by Open Source Experts
Camptocamp
Comments
  • Missing results following scan of bucket

    Missing results following scan of bucket

    I have followed instructions for launching terraboard from docker, and last week was able to display results. However, while preparing to demo to my team today, my results are unavailable. Here is partial log of run:

    docker logs terraboard
    Loading config from /temp/thiscorp.yaml
    time="2022-04-12T05:44:18Z" level=info msg="Terraboard vv2.1.1 (built for Terraform v1.0.2) is starting..."
    time="2022-04-12T05:44:18Z" level=info msg="Using AWS (S3+DynamoDB) as state/locks provider"
    time="2022-04-12T05:44:18Z" level=info msg=Automigrate
    time="2022-04-12T05:44:18Z" level=info msg="Refreshing DB"
    time="2022-04-12T05:44:18Z" level=info msg="Serving swagger on port 8081"
    
    

    I'm pretty sure the large state files are an issue

  • Compatibility Issue with Terraform v0.12.0

    Compatibility Issue with Terraform v0.12.0

    Once I upgrade Terraform version to v0.12.0(provider.aws v2.10.0) and "terraform apply" to update state files, then state files disappear in the main dashboard page. Terraborad still can detect that those state files are v0.12.0 but won't display resources within the files. Could someone suggest, please?

  • Config file support

    Config file support

    I simply add the option of using a YAML file for configuring the service. Configurations loaded from file have the highest priority over others methods (cli params, env variables).

    My idea is made multiples PR for achieving the multiple bucket configuration feature.

  • feat: add plan db scheme

    feat: add plan db scheme

    I used the db scheme available in the TerraVerge repository to which I added a foreign key to the associated bucket (I thought it would be useful if we add support for several simultaneous bucket) and the CreatedAt / UpdatedAt columns

    Here is the table that it creates (visualized from pgadmin): Capture d’écran de 2021-06-07 12-05-45

  • Authentication info

    Authentication info

    This is a WIP to display authentication info when an auth proxy is used.

    The current state of the PoC:

    • checks for an X-Forwarded-Access-Token header, gets info from GitHub when passed (only Github is supported so far)
    • If no X-Forwarded-Access-Token header is passed, uses X-Forwarded-User and X-Forwarded-Email, using gravatar for the avatar URL.
  • GCS support

    GCS support

    A first attempt at adding GCS support to Terraboard. Allows authentication either using GOOGLE_APPLICATION_CREDENTIALS or by mounting a service account key and specifying the path as an argument.

    I tried to match S3 design as much as possible.

    Requirements:

    • Versioning should be enabled on the bucket
    • Service account must have read access on the specified bucket

    Tested:

    • Was able to import versioned state files from GCS and view diffs between them as normal
    • Connecting both using default credentials and service accounts.

    Should still be tested:

    • Proper display of locked states. Did not have a good setup for testing this.
  • Support Kubernetes Deployment

    Support Kubernetes Deployment

    GIven it is a dockerized project, pretty easy to deploy on K8S as well with the right configurations. I have done it my self but IMO should be supported as out of the box as well.

  • Use S3 compatible bucket

    Use S3 compatible bucket

    Hi,

    First of all thank for developing terraboard, it's a really nice tool!

    We are using our own S3 compatible system and currently I think can't change the address with AWS_BUCKET environment variable.

    I would like to be able to use AWS_BUCKET=s3://s3.offline.net/bucket_xxx. Maybe I missed something but I think AWS addresses are deeply linked to aws-sdk-go.

  • Added Minio as a private alternative to AWS S3

    Added Minio as a private alternative to AWS S3

    We required Minio as a state backend. As it does not store file versions by default and has also a slightly different handling than AWS S3, I decided to write my own "state provider". File versioning and locking might be added in the future, but is not yet implemented.

    Signed-off-by: Tim Beermann [email protected]

  • v0.21.0 can't connect to RDS db when running on AWS ECS

    v0.21.0 can't connect to RDS db when running on AWS ECS

    background: Running postgres on an RDS instance Trying to run terraboard on ECS (EC2 instance)

    When I try to run terraboard's docker image on AWS ECS I get an error on startup that the RDS DB instance is busy

    level=info msg="Terraboard v0.21.0 (built for Terraform v0.12.28) is starting..."
    level=info msg="Using AWS (S3+DynamoDB) as the state/locks provider"
    level=fatal msg="dial tcp: lookup terraboard.abcd1234.us-east-1.rds.amazonaws.com: device or resource busy"
    

    I confirmed all my configs/permissions are correct by running the container locally with no issues, also confirmed that the ECS instance the container runs on is able to connect to the RDS instance directly so that end is setup correctly as well.

  • Does not work with AWS cross-account assumed role

    Does not work with AWS cross-account assumed role

    Hi.

    I have my cli configured with its token id and secret key. It has permission to assume a role in another AWS account, where the s3 bucket and dynamo table are located.

    How do I configure terraboard for this setup? Simply passing AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID of the linked account to the terraboard image does not work of course.

    The goal here is to have one terraboard showcasing multiplestate files from multiple aws accounts. That would require setting multiple dynamodb tables. Is this possible?

    Thanks in advance.

  • Feat/add Tencent Cloud Object Storage (COS) support

    Feat/add Tencent Cloud Object Storage (COS) support

    Hi @hbollon , I've updated my codes based on your comments in previous PR(https://github.com/camptocamp/terraboard/pull/255). 1.added UT for COS. 2.updated document. make update-docs update the README.md 3.removed the commented code.

    Pls start your review if you are free :)

  • Support for IRSA

    Support for IRSA

    Currently there are two ways to configure the app, either with an IAM user (key and secret) or by supplying a role, perhaps in addition to the IAM user.

    This practice is becoming outdated in favor of using an IAM role for service account (IRSA), at least with AWS EKS clusters. Would it be possible to leave the configuration of the identity to the underlying system? It should be less code. Following the IRSA setup steps, one could deploy a pod which has an IAM role as its identity and the Go SDK should just pick up on it.

    The code around here would have to be changed a bit. https://github.com/camptocamp/terraboard/blob/056ca947f4cdff34d903108e59b4d089e7367f1e/state/aws.go#L45-L58

    See https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/

  • Is Support for HTTP Backend Planned?

    Is Support for HTTP Backend Planned?

    I've been looking at this project, as it looks super interesting for a selfhosted Terraform solution, but currently since I'm not sure if I will be using S3 buckets or an HTTP backend like terraform-http-backend for my state, I was wondering if support for the HTTP backend is planned? Thanks!

  • Is Terraboard supposed to delete old state files?

    Is Terraboard supposed to delete old state files?

    Sorry, can't seem to find where I can post a question.

    I would like to know if the terraboard process is supposed to delete data in the tables that contain the results of the query from the state files. I have deleted .tfstate files in my S3 bucket but it seems that records are still in the terraboard db.

    Apologies if this question here is completely misplaced

  • AWS Default Credentials Chain not fetched in > 2.0.0

    AWS Default Credentials Chain not fetched in > 2.0.0

    As of this PR: https://github.com/camptocamp/terraboard/pull/177

    AWS Default Credentials Provider is never used so now we're forced to either pass in an access key/secret key or pass in a role to assume. However, the default credentials provider is much more flexible: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/#specifying-credentials

    In our case, we use web identity auth which is now broken.

  • Allow Binding To Localhost Only

    Allow Binding To Localhost Only

    Currently Terraboard binds to all interfaces at start and there is no option to be able to bind to 127.0.0.1 only. In order to make it such that you can't bypass oauth2 proxy if it is in use, it would be nice to have this configurable so that terraboard is not accessible externally to the container.

β›‘ Gatus - Automated service health dashboard
β›‘ Gatus - Automated service health dashboard

A service health dashboard in Go that is meant to be used as a docker image with a custom configuration file. I personally deploy it in my Kubernetes

Dec 31, 2022
This Simple script is used to convert Datadog Dashboard to NewRelic.
This Simple script is used to convert Datadog Dashboard to NewRelic.

What is this? This Simple script is used to convert Datadog Dashboard to NewRelic. This script is build with specific dashboard layout in mind, so it

Feb 6, 2022
Terraform Algolia Provider

Terraform Provider Algolia Terraform Provider for Algolia. Documentation Full, comprehensive documentation is available on the Terraform

Dec 14, 2022
A terraform provider for Sparkpost

terraform-provider-sparkpost A terraform provider for Sparkpost Local Development Run the following command to build the provider make build Test Exam

Mar 1, 2022
Terraform Provider for cascading runs across multiple workspaces.

Terraform Multispace Provider The multispace Terraform provider implements resources to help work with multi-workspace workflows in Terraform Cloud (o

Oct 25, 2022
Terraform Provider for PGP Actions

Terraform Provider PGP Warning: Use of this provider will result in secrets being in terraform state in PLAIN TEXT (aka NOT ENCRYPTED). You've been wa

Sep 30, 2022
OPG sirius supervision firm deputy hub: Managed by opg-org-infra & Terraform

OPG sirius supervision firm deputy hub: Managed by opg-org-infra & Terraform

Jan 10, 2022
Terraform provider for OCM

Terraform provider for OCM Build To build the provider use the make command. Use To use the provider first build and install it: $ make install Then g

Nov 11, 2021
Pulumi Terraform provider for Artifactory

Terraform Bridge Provider Boilerplate This repository contains boilerplate code for building a new Pulumi provider which wraps an existing Terraform p

Aug 11, 2022
Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Antch Antch, inspired by Scrapy. If you're familiar with scrapy, you can quickly get started. Antch is a fast, powerful and extensible web crawling &

Jan 6, 2023
A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

Dec 17, 2022
Nodebook - Multi-Lang Web REPL + CLI Code runner
Nodebook - Multi-Lang Web REPL + CLI Code runner

nodebook Nodebook - Multi-Language REPL with Web UI + CLI code runner Useful to practice algorithms and datastructures for coding interviews. What is

Dec 29, 2022
A simple web shop that accepts TurtleCoin
A simple web shop that accepts TurtleCoin

TurtleShop A simple web shop that accepts TurtleCoin Note: This software is not finished. Donate to help development of TurtleShop TRTLuxEnfjdF46cBoHh

Feb 13, 2022
this is an example of hystrix-go usage in web dev

hystrix-go-example this is an example of hystrix-go usage in web dev Explanation this example contains 2 service: alpha as our main service, circuit b

Apr 22, 2022
Http web frame with Go Programming Language

Http web frame with Go Programming Language

Oct 17, 2021
Scrape the web in the eink era. Convert websites into books.

Scrape the web in the eink era. Convert websites into books.

Dec 29, 2022
A small API to be used in Traefik Forward Auth to block attacks on services behind your favorite web router

Dynamic IPs Blacklist for Traefik This small software implements the possibility to block access to all the services behind your Traefik service. The

Sep 2, 2022
Go client library for Geonames Web Services (geonames.org)

geonames Go client library for Geonames Web Services (geonames.org) Usage Register your username at geonames.org Export ENV Var or read the username f

Nov 23, 2022
A Simple Bank Web Service implemented in Go, HTTP & GRPC, PostgreSQL, Docker, Kubernetes, GitHub Actions CI

simple-bank Based on this Backend Master Class by TECH SCHOOL: https://youtube.com/playlist?list=PLy_6D98if3ULEtXtNSY_2qN21VCKgoQAE Requirements Insta

Dec 9, 2021