An open source alternative to terraform enterprise.

oTF

An open source alternative to terraform enterprise.

Functionality is currently limited:

  • Remote execution mode (plans and applies run remotely)
  • State backend (state stored in postgresql)
  • Workspace management (supports terraform workspace commands)
  • No web frontend; CLI/API support only.

Getting Started

These steps will get you started with running everything on your local system. You'll setup the server, configure SSL so that terraform trusts the server, and then configure terraform. You'll then be able to run terraform commands using the server as a remote backend.

demo

  1. Download a release. The zip file contains two binaries: a daemon and a client, otfd and otf. Extract them to a directory in your PATH, e.g. /usr/local/bin.

  2. Generate SSL cert and key. For example, to generate a self-signed cert and key for localhost using openssl:

    openssl req -x509 -newkey rsa:4096 -sha256 -keyout key.pem -out cert.crt -days 365 -nodes -subj '/CN=localhost' -addext 'subjectAltName=DNS:localhost'

  3. Ensure your system trusts the generated cert. For example, on Linux:

    sudo cp cert.crt /usr/local/share/ca-certificates
sudo update-ca-certificates

  4. Ensure you have access to a postgresql server. oTF assumes it's running locally on a unix domain socket in /var/run/postgresql. Create a database named otf:

    createdb otfd

  5. Run the oTF daemon:

    otfd --ssl --cert-file=cert.crt --key-file=key.pem

    The daemon runs in the foreground and can be left to run.

    Note: you can customise the postgres connection string by passing it via the flag --database.

  6. In another terminal, login to your OTF server (this merely adds some dummy credentials to ~/.terraform.d/credentials.tfrc.json):

    otf login

  7. Configure the terraform backend and define a resource:

    cat > main.tf <<EOF
terraform {
  backend "remote" {
    hostname = "localhost:8080"
    organization = "default"

    workspaces {
      name = "dev"
    }
  }
}

resource "null_resource" "e2e" {}
EOF

  8. Run terraform!:

    terraform init
terraform plan
terraform apply

Next Steps

oTF is a mere prototype but a roadmap of further features is planned:

  • User AuthN/Z
  • Agents
  • Terminal application
  • Github integration
  • Policies (OPA?)
  • Web frontend

Building

You'll need Go installed.

Clone the repo, and then build and install the binary using the make task:

git clone https://github.com/leg100/otf
cd otf
make install

That'll create a binary inside your go bins directory (defaults to $HOME/go/bin).

Owner
Louis Garman
All things Go, Terraform, and Kubernetes.
Louis Garman
https://github.com/leg100/otf
Comments
  • Return user to original URL once logged in

    Return user to original URL once logged in

    Currently, when a user with invalid/missing session visits a web app URL, they are redirected to the login prompt. Once successfully logged in they are simply sent straight to their profile page.

    Fix this so that they are instead returned the original URL they tried to access.

  • workspace list error

    workspace list error 

    ✦ > _build/otf workspaces list --organization automatize
panic: reflect.Set: value of type *otf.Pagination is not assignable to type *dto.Pagination

goroutine 1 [running]:
reflect.Value.assignTo({0x9f4240?, 0xc0002b6000?, 0xc0002b6000?}, {0xabeef9, 0xb}, 0x9f4a40, 0x0)
	/usr/local/go/src/reflect/value.go:3062 +0x2ac
reflect.Value.Set({0x9f4a40?, 0xc000260120?, 0x0?}, {0x9f4240?, 0xc0002b6000?, 0x0?})
	/usr/local/go/src/reflect/value.go:2088 +0xeb
github.com/leg100/otf/http.unmarshalResponse({0xbbfea0?, 0xc000038080}, {0x9f4e40, 0xc000260120?})
	/home/louis/co/otf/http/client.go:412 +0x4c6
github.com/leg100/otf/http.(*client).do(0xc0000cc6e0, {0xbc4ab0?, 0xc00021e080}, 0xc00025e100, {0x9f4e40, 0xc000260120})
	/home/louis/co/otf/http/client.go:287 +0x4c5
github.com/leg100/otf/http.(*workspaces).List(0xc000266078, {0xbc4ab0, 0xc00021e080}, {{0x0, 0x0}, {0x0, 0x0}, 0xc0000fd1a0, 0x0})
	/home/louis/co/otf/http/workspace_client.go:79 +0x185
main.WorkspaceListCommand.func1(0xc000192a00, {0xabbb14?, 0x2?, 0x2?})
	/home/louis/co/otf/cmd/otf/workspace_list.go:25 +0x8d
github.com/spf13/cobra.(*Command).execute(0xc000192a00, {0xc00007afa0, 0x2, 0x2})
	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0xc00015db80)
	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x39c
github.com/spf13/cobra.(*Command).Execute(...)
	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:890
main.Run({0xbc4ab0?, 0xc00021e080}, {0xc00001e0b0, 0x4, 0x4})
	/home/louis/co/otf/cmd/otf/main.go:51 +0x253
main.main()
	/home/louis/co/otf/cmd/otf/main.go:17 +0x85
  • Remove irrelevant output from terraform plan

    Remove irrelevant output from terraform plan

    A terraform plan via oTF prints the following text at the end:

    Terraform will perform the following actions:

  # null_resource.demo will be created
  + resource "null_resource" "demo" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.out

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.out"

    The last few lines of output are irrelevant and should be removed before being streamed to the client.

  • Resource attributes should be pointers

    Resource attributes should be pointers

    They should be *int but oTF is using int. This is what TFC returns for an unreachable apply:

    {
  "data": {
    "id": "apply-ejszsaRdfEabKsuu",
    "type": "applies",
    "attributes": {
      "status": "unreachable",
      "status-timestamps": {},
      "log-read-url": "https://archivist.terraform.io/v1/object/dmF1bHQ6djE6Zi83aHZYdC9xVFdnWTJHdWYrNHJrOE9MWWtTWElIL2tyaWhkcXhNcEhzWENra2gxRDk0cGZFQ1F1a2Nn
b1ppVGVJbGRhcU1YdGpHalhGT2kxeEJDWjdoUmc4UVBOQ1lxTHRWRVl5UmxjK1JxQ3E5dGVRVG1TUlhXWnZlcG9MZ0N5M014U2hBNVZEY1FZL1JhdWI2aCtRRHBuNXhkY3BhMGxtV290MkdjSkFzMUlaZzlM
VFgzam5JTjIvd2Z0NzlOTno5R3c5b2hrQ3dnWUJGVTNhK1YvdU9DL2JjcUN0RzU5bVZYcGtaZmVGeC9YSTNxYXFkREFuZGdzd0VxR0ZyY2h3SkcrU2xKZnM4TnlwR1ZjcmI0TkhvMnBkT0x6YzFrczV0Tyt0
Y1NVODJqVE8vYzZmd3ZSMDNnNEp3Q3A0TXRDVUoybFZZWlkzSm9JYTVqWk90VTZVb2hCUUlSaUpxMFNRMk9zTklKd0dzVnF6NEVFS1gvcjB1eFEyVEN6N2tp",
      "resource-additions": null,
      "resource-changes": null,
      "resource-destructions": null,
      "structured-run-output-enabled": false,
      "execution-details": null
    },
    "relationships": {
      "state-versions": {
        "data": []
      }
    },
    "links": {
      "self": "/api/v2/applies/apply-ejszsaRdfEabKsuu"
    }
  }
}
  • Honor terraform version

    Honor terraform version

    oTF just uses whatever version of terraform is in its PATH, ignoring both the:

    • default version specified in the codebase
    • per-workspace version specified in workspace obj

    It should honor both. It should probably download the specified version too if not found.

  • Workspace webhook

    Workspace webhook

    When connecting a workspace to a VCS repo, add a webhook to the repo subscribing to push/commit events on the default branch.

    This is a pre-requisite to the greater goal of triggering runs via commits.

  • Manage VCS Permission

    Manage VCS Permission

    Add an organization-wide permission to manage VCS providers, as per TFC:

    https://developer.hashicorp.com/terraform/cloud-docs/users-teams-organizations/permissions#manage-vcs-settings

  • Make config max size configurable

    Make config max size configurable

    Current max limit of a config tarball is hardcoded to 10mb.

    Make it user settable via flag etc.

    Also: it's only checked on http handler. It needs to be checked instead at the service layer so that vcs tarballs are checked too.

  • Working directory option

    Working directory option

    Add ability to set a working directory for a workspace:

    https://developer.hashicorp.com/terraform/cloud-docs/workspaces/settings#terraform-working-directory

  • Prohibit CLI apply on connected workspace

    Prohibit CLI apply on connected workspace

    OTF should replicate TFC behaviour of prohibiting an apply via the CLI on a workspace connected to a VCS repo, reporting an error like so:

    > terraform apply
╷
│ Error: Apply not allowed for workspaces with a VCS connection
│
│ A workspace that is connected to a VCS requires the VCS-driven workflow to ensure that the VCS remains the single source of truth.
╵
  • Resolve or silence error regarding organization notification

    Resolve or silence error regarding organization notification

    Harmless error keeps popping up in logs:

    2022-11-19 00:09:26 | ERR unmarshaling postgres notification error="unknown table specified in events notification: organization" component=pubsub
Related tags
DevOps Tools terraform
Terraform-operator - The Terraform Operator provides support to run Terraform modules in Kubernetes in a declaritive way as a Kubernetes manifest

Terraform Operator The Terraform Operator provides support to run Terraform modu

Oct 19, 2022
Terraform-equinix-migration-tool - Tool to migrate code from Equinix Metal terraform provider to Equinix terraform provider

Equinix Terraform Provider Migration Tool This tool targets a terraform working

Feb 15, 2022
Terraform-in-Terraform: Execute Modules directly from the Terraform Registry

Terraform-In-Terraform Provider This provider allows running Terraform in Terraform. This might seem insane but there are some edge cases where it com

Dec 25, 2022
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers
 An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

Oct 19, 2021
The open source public cloud platform. An AWS alternative for the next generation of developers.
 The open source public cloud platform. An AWS alternative for the next generation of developers.

M3O M3O is an open source public cloud platform. We are building an AWS alternative for the next generation of developers. Overview AWS was a first ge

Jan 2, 2023
Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

Terraform Website: https://www.terraform.io Forums: HashiCorp Discuss Documentation: https://www.terraform.io/docs/ Tutorials: HashiCorp's Learn Platf

Jan 9, 2023
go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data.
 go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data.

go-opa-validate go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data. Installation Usage Cont

Nov 17, 2022
Terraform utility provider for constructing bash scripts that use data from a Terraform module

Terraform Bash Provider This is a Terraform utility provider which aims to robustly generate Bash scripts which refer to data that originated in Terra

Sep 6, 2022
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)
 Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Dec 8, 2022
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)
 Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)

Terraform Provider for Azure (Resource Manager) Version 2.x of the AzureRM Provider requires Terraform 0.12.x and later, but 1.0 is recommended. Terra

Oct 16, 2021
Quick start repository for creating a Terraform provider using terraform-plugin-framework

Terraform Provider Scaffolding (Terraform Plugin Framework) This template repository is built on the Terraform Plugin Framework. The template reposito

Dec 15, 2022
Terraform-provider-mailcow - Terraform provider for Mailcow

Terraform Provider Scaffolding (Terraform Plugin SDK) This template repository i

Dec 31, 2021
Terraform-provider-buddy - Terraform Buddy provider For golang

Terraform Provider for Buddy Documentation Requirements Terraform >= 1.0.11 Go >

Jan 5, 2022
Terraform-provider-vercel - Terraform Vercel Provider With Golang

Vercel Terraform Provider Website: https://www.terraform.io Documentation: https

Dec 14, 2022
Terraform-grafana-dashboard - Grafana dashboard Terraform module

terraform-grafana-dashboard terraform-grafana-dashboard for project Requirements

May 2, 2022
Puccini-terraform - Enable TOSCA for Terraform using Puccini

(work in progress) TOSCA for Terraform Enable TOSCA for Terraform using Puccini.

Jun 27, 2022
Terraform Provider Scaffolding (Terraform Plugin SDK)

Terraform Provider Scaffolding (Terraform Plugin SDK) This template repository is built on the Terraform Plugin SDK. The template repository built on

Feb 8, 2022
Terraform-ncloud-docs - Terraform-ncloud-docs

terraform-ncloud-docs Overview This docs help to use terraform creation server C

Oct 2, 2022
Terraform-provider-age - Age Terraform Provider with golang

Age Terraform Provider This provider lets you generate an Age key pair. Using th

Feb 15, 2022