OTS: Open Terraforming Server

OTS: Open Terraforming Server

A prototype open source alternative to terraform enterprise.

Functionality is currently limited:

  • State backend (state stored in a sqlite database)
  • Workspace management (supports terraform workspace commands)
  • Local execution mode (plans and applies run locally)

Getting Started

These steps will get you started with running everything on your local system. You'll setup the server, configure SSL so that terraform trusts the server, and then configure terraform. You'll then be able to run terraform commands using the server as a remote backend.

demo

  1. Download and extract a release.

  2. Generate SSL cert and key. For example, to generate a self-signed cert and key for localhost:

    openssl req -x509 -newkey rsa:4096 -sha256 -keyout key.pem -out cert.crt -days 365 -nodes -subj '/CN=localhost' -addext 'subjectAltName=DNS:localhost'
  3. Ensure your system trusts the generated cert. For example, on Linux:

    sudo cp cert.crt /usr/local/share/ca-certificates
    sudo update-ca-certificates
    
  4. Run the OTS daemon:

    ./otsd -ssl -cert-file cert.crt -key-file key.pem

    The daemon runs in the foreground and can be left to run.

  5. In another terminal create an organization:

    curl -H"Accept: application/vnd.api+json" https://localhost:8080/api/v2/organizations -d'{
      "data": {
        "type": "organizations",
        "attributes": {
          "name": "mycorp",
          "email": "[email protected]"
        }
      }
    }'
  6. Enter some dummy credentials (this is necessary otherwise terraform will complain):

    cat > ~/.terraform.d/credentials.tfrc.json <<EOF
    {
      "credentials": {
        "localhost:8080": {
          "token": "dummy"
        }
      }
    }
    EOF
  7. Configure the terraform backend and define a resource:

    cat > main.tf <<EOF
    terraform {
      backend "remote" {
        hostname = "localhost:8080"
        organization = "mycorp"
    
        workspaces {
          name = "dev"
        }
      }
    }
    
    resource "null_resource" "e2e" {}
    EOF
  8. Run terraform!:

    terraform init
    terraform plan
    terraform apply

Next Steps

OTS is a mere prototype but a roadmap of further features could be:

  • User AuthN/Z
  • Remote execution mode
  • Agents
  • Github integration
  • Policies (OPA?)
  • Web frontend

Building

You'll need Go installed.

Clone the repo, and then build and install the binary using the make task:

git clone https://github.com/leg100/ots
cd ots
make install

That'll create a binary inside your go bins directory (defaults to $HOME/go/bin).

Owner
Louis Garman
All things Kubernetes and Terraform.
Louis Garman
Comments
  • Return user to original URL once logged in

    Return user to original URL once logged in

    Currently, when a user with invalid/missing session visits a web app URL, they are redirected to the login prompt. Once successfully logged in they are simply sent straight to their profile page.

    Fix this so that they are instead returned the original URL they tried to access.

  • workspace list error

    workspace list error

    ✦ > _build/otf workspaces list --organization automatize
    panic: reflect.Set: value of type *otf.Pagination is not assignable to type *dto.Pagination
    
    goroutine 1 [running]:
    reflect.Value.assignTo({0x9f4240?, 0xc0002b6000?, 0xc0002b6000?}, {0xabeef9, 0xb}, 0x9f4a40, 0x0)
    	/usr/local/go/src/reflect/value.go:3062 +0x2ac
    reflect.Value.Set({0x9f4a40?, 0xc000260120?, 0x0?}, {0x9f4240?, 0xc0002b6000?, 0x0?})
    	/usr/local/go/src/reflect/value.go:2088 +0xeb
    github.com/leg100/otf/http.unmarshalResponse({0xbbfea0?, 0xc000038080}, {0x9f4e40, 0xc000260120?})
    	/home/louis/co/otf/http/client.go:412 +0x4c6
    github.com/leg100/otf/http.(*client).do(0xc0000cc6e0, {0xbc4ab0?, 0xc00021e080}, 0xc00025e100, {0x9f4e40, 0xc000260120})
    	/home/louis/co/otf/http/client.go:287 +0x4c5
    github.com/leg100/otf/http.(*workspaces).List(0xc000266078, {0xbc4ab0, 0xc00021e080}, {{0x0, 0x0}, {0x0, 0x0}, 0xc0000fd1a0, 0x0})
    	/home/louis/co/otf/http/workspace_client.go:79 +0x185
    main.WorkspaceListCommand.func1(0xc000192a00, {0xabbb14?, 0x2?, 0x2?})
    	/home/louis/co/otf/cmd/otf/workspace_list.go:25 +0x8d
    github.com/spf13/cobra.(*Command).execute(0xc000192a00, {0xc00007afa0, 0x2, 0x2})
    	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x67c
    github.com/spf13/cobra.(*Command).ExecuteC(0xc00015db80)
    	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x39c
    github.com/spf13/cobra.(*Command).Execute(...)
    	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
    github.com/spf13/cobra.(*Command).ExecuteContext(...)
    	/home/louis/go/pkg/mod/github.com/spf13/[email protected]/command.go:890
    main.Run({0xbc4ab0?, 0xc00021e080}, {0xc00001e0b0, 0x4, 0x4})
    	/home/louis/co/otf/cmd/otf/main.go:51 +0x253
    main.main()
    	/home/louis/co/otf/cmd/otf/main.go:17 +0x85
    
  • Remove irrelevant output from terraform plan

    Remove irrelevant output from terraform plan

    A terraform plan via oTF prints the following text at the end:

    Terraform will perform the following actions:
    
      # null_resource.demo will be created
      + resource "null_resource" "demo" {
          + id = (known after apply)
        }
    
    Plan: 1 to add, 0 to change, 0 to destroy.
    
    ─────────────────────────────────────────────────────────────────────────────
    
    Saved the plan to: plan.out
    
    To perform exactly these actions, run the following command to apply:
        terraform apply "plan.out"
    

    The last few lines of output are irrelevant and should be removed before being streamed to the client.

  • Resource attributes should be pointers

    Resource attributes should be pointers

    They should be *int but oTF is using int. This is what TFC returns for an unreachable apply:

    {
      "data": {
        "id": "apply-ejszsaRdfEabKsuu",
        "type": "applies",
        "attributes": {
          "status": "unreachable",
          "status-timestamps": {},
          "log-read-url": "https://archivist.terraform.io/v1/object/dmF1bHQ6djE6Zi83aHZYdC9xVFdnWTJHdWYrNHJrOE9MWWtTWElIL2tyaWhkcXhNcEhzWENra2gxRDk0cGZFQ1F1a2Nn
    b1ppVGVJbGRhcU1YdGpHalhGT2kxeEJDWjdoUmc4UVBOQ1lxTHRWRVl5UmxjK1JxQ3E5dGVRVG1TUlhXWnZlcG9MZ0N5M014U2hBNVZEY1FZL1JhdWI2aCtRRHBuNXhkY3BhMGxtV290MkdjSkFzMUlaZzlM
    VFgzam5JTjIvd2Z0NzlOTno5R3c5b2hrQ3dnWUJGVTNhK1YvdU9DL2JjcUN0RzU5bVZYcGtaZmVGeC9YSTNxYXFkREFuZGdzd0VxR0ZyY2h3SkcrU2xKZnM4TnlwR1ZjcmI0TkhvMnBkT0x6YzFrczV0Tyt0
    Y1NVODJqVE8vYzZmd3ZSMDNnNEp3Q3A0TXRDVUoybFZZWlkzSm9JYTVqWk90VTZVb2hCUUlSaUpxMFNRMk9zTklKd0dzVnF6NEVFS1gvcjB1eFEyVEN6N2tp",
          "resource-additions": null,
          "resource-changes": null,
          "resource-destructions": null,
          "structured-run-output-enabled": false,
          "execution-details": null
        },
        "relationships": {
          "state-versions": {
            "data": []
          }
        },
        "links": {
          "self": "/api/v2/applies/apply-ejszsaRdfEabKsuu"
        }
      }
    }
    
  • Honor terraform version

    Honor terraform version

    oTF just uses whatever version of terraform is in its PATH, ignoring both the:

    • default version specified in the codebase
    • per-workspace version specified in workspace obj

    It should honor both. It should probably download the specified version too if not found.

  • Working directory option

    Working directory option

    Add ability to set a working directory for a workspace:

    https://developer.hashicorp.com/terraform/cloud-docs/workspaces/settings#terraform-working-directory

  • Prohibit CLI apply on connected workspace

    Prohibit CLI apply on connected workspace

    OTF should replicate TFC behaviour of prohibiting an apply via the CLI on a workspace connected to a VCS repo, reporting an error like so:

    > terraform apply
    ╷
    │ Error: Apply not allowed for workspaces with a VCS connection
    │
    │ A workspace that is connected to a VCS requires the VCS-driven workflow to ensure that the VCS remains the single source of truth.
    ╵
    
  • Resolve or silence error regarding organization notification

    Resolve or silence error regarding organization notification

    Harmless error keeps popping up in logs:

    2022-11-19 00:09:26 | ERR unmarshaling postgres notification error="unknown table specified in events notification: organization" component=pubsub
    
Neko is a cross-platform open-source animated cursor-chasing cat. This is the reimplementation write in Go.

Neko Neko is a cat that chases the mouse cursor across the screen, an app written in the late 1980s and ported for many platforms. This code is a re-i

Nov 21, 2022
A bin which will keep screen open by moving a mouse

Stay Awake This is a small program which will move mouse up and down to keep screen on. This stimulates like user is doing something. Motivation I had

Oct 21, 2021
Code snippets by first time open source contributors

Introduction Golang code snippets by first time open source contributors Rules How to contribute Add a folder and create your desired code snippet fil

Oct 6, 2021
QuickAbeHiroshi - The golang program to open a web site which is the fastest in the world
QuickAbeHiroshi - The golang program to open a web site which is the fastest in the world

The golang program to open a web site which is the fastest in the world.

Jan 2, 2022
Tidb - An open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads
Tidb - An open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads

What is TiDB? TiDB ("Ti" stands for Titanium) is an open-source NewSQL database

Jan 5, 2022
Prometheus statistics exporter for Open vSwitch

Prometheus statistics exporter for Open vSwitch Open vSwitch is popular virutal switch that enables high performance software defined networking. Sinc

Feb 18, 2022
RTS: request to struct. Generates Go structs from JSON server responses.

RTS: Request to Struct Generate Go structs definitions from JSON server responses. RTS defines type names using the specified lines in the route file

Dec 7, 2022
A minimalistic message queue server, in Go

lineup, a tiny queue server lineup is a very simple queue server. all it does is serve a binary-safe priority queue with a simple protocol insp

Aug 17, 2022
ephemeral metric aggregation server

hemera hemera is a zero-dependency statsd implementation written in Go. The project takes advantage of modular packages and interfaces to make connect

Jun 13, 2020
server for dozenplans
server for dozenplans

DozenPlans Server 学习养成计划主要是针对考研学生用户的任务管理系统, 用户可以添加 学习 任务, 每个任务都有自己的主题、 完成时间和优先级, 根据设定的完成时间和优先级对 用户进行不同频率的提醒, 直到其完成该任务, 任务可以设定提醒频率和时间期限, 避免导致任务越堆积越多, 除

Sep 17, 2021
thermal print server for esc pos printers

thprint thprint is an extremely simple printing server for thermal printers connected to raspberry pis. This printing server exposes a print endpoint

Nov 4, 2021
Count Dracula is a fast metrics server that counts entries while automatically expiring old ones

In-Memory Expirable Key Counter This is a fast metrics server, ideal for tracking throttling. Put values to the server, and then count them. Values ex

Jun 17, 2022
Module to ease interaction with Pact's development server & ScalableBFT

go-pact Module to ease interaction with Pact's development server & ScalableBFT Install go install github.com/jfamousket/go-pact@latest Functions H

Dec 9, 2021
Example ReSTFul Server in Golang - SBU-Snapp-School

students.go Introduction ReSTful API to create, update, retrieve and delete students from a datastore. Requests needs validation, package needs tests,

Sep 14, 2022
Phalanx is a cloud-native full-text search and indexing server written in Go built on top of Bluge that provides endpoints through gRPC and traditional RESTful API.

Phalanx Phalanx is a cloud-native full-text search and indexing server written in Go built on top of Bluge that provides endpoints through gRPC and tr

Dec 25, 2022
A server for TurboRepo Remote Cache to store cache artefacts in Google Cloud Storage or Amazon S3

Tapico Turborepo Remote Cache This is an implementation of Vercel's Turborepo Remote Cache API endpoints used by the turborepo CLI command. This solut

Dec 13, 2022
Temporal Server docker-compose files

Temporal Server docker-compose files This repository provides docker-compose fil

Dec 24, 2021
Transcoder - This package implements a transcoding server

muxable/transcoder This package implements a transcoding server. It accepts WebR

Dec 30, 2022
Embedded javascript server-side renderer for Golang

v8ssr Embedded javascript server-side renderer for Golang. Useful for static server-side rendering. This does not attempt to polyfill node or browser

Aug 27, 2022