Go Hosting Solution for AWS, Google Could and Digital Ocean

Furnace

Logo

Go Report Card Build Status Coverage Status Awesome

Intro

Brief Explanation

Here is a very short description of what Furnace does in a handy IKEA manual format.

Furnace1 Furnace2

In More Depth

AWS Cloud Formation, Google Cloud Platform, or DigitalOcean hosting with Go. This project utilizes the power of AWS CloudFormation and CodeDeploy, or DeploymentManager and Git support in GCP in order to simply deploy an application into a robust, self-healing, redundant environment. The environment is configurable through the CloudFormation Template or GCPs jinja files. A sample can be found in the templates folder.

The application to be deployed is handled via GitHub, or S3.

A sample application is provider under the furnace-codedeploy-app folder.

Furnace vs Terraform

Furnace does not try to compete with Terraform. It aims for a different market. The main differences between Terraform and Furnace are the following:

Binary size

On the rare occasions when disk space matters, Furnace provides individual binaries. The size of the aws binary ATM is 15MB. Terraform is at 100MB.

Configuration

Configuration for Terraform can be pretty huge. In contrast Furnace's configuration is lightweight because frankly, it doesn't have much. All the configuration you will have to write will be for CloudFormation and GCP. However complicated they can be is out of Furnace's reach.

Vendor Lock

True that Terraform provides provider agnostic settings and behavior. But, you'll be vendor locked to Terraform. Moving away from the massive configuration that the user has to build up to use it can never be moved away from again. Or only through a lot of work.

In contrast, Furance is a light wrapper around services that provide what Terraform is providing per provider. What does this mean? It means Furnace is using CloudFormation for AWS and DeploymentManager for GCP which are services built by AWS and GCP. Not by Furnace. If you don't want to use Furnace any longer, you'd still have your deployment configuration which works just fine without it. Resources are all grouped together. Deleting them is as simple as calling an end-point, clicking a button or hitting enter.

Installing Binaries

Go Install

To install all generated binaries at once, run:

# Download / Clone the latest version
# cd into go-furnace
make install-all

This will install all dependencies and both binaries to $GOPATH/bin folder.

Make commands

You can also build the commands which will be output into the cmd sub-folder.

# Simply run make from the root folder
make

Building for different environment

Convenient targets are provided for linux and windows binaries.

make linux
make windows

These are only available from the package folders respectively.

Clean

In case make install is used, a clean-up target is also provided.

make clean-all

AWS

Google Cloud

DigitalOcean

Plugins

A highly customizable plugin system is provided for Furnace via HashiCorp's Go-Plugins.

Writing a plugin is as easy as implementing an interface. Furnace uses GRPC to talk to the plugins locally. The interface to implement is provided by a proto file located here: Protocol Description.

A single configuration value is provided for plugins in the yaml file which is the location of plugins:

  plugins:
    plugin_path: "./plugins"

If this is not provided, the default value is ./plugins which is next to the binary.

Plugins are available for the following events:

  • Pre creating a stack (stackname parameter is provided) These plugins have the chance to stop the process before it starts. Here the user would typically try and do a preliminary check like permissions or resources are available. If not, abort the creation process before it begins.

  • Post creating a stack (stackname parameter is provided) This is typically a place where a post notification could be executed, like a slack notifier that a stack's creation is done. Or an application health-check which looks up the deployed URL parameter and checks if the application is responding.

  • Pre deleting a stack (stackname parameter is provided) These plugins also have the option to abort a delete before it begins. A typical use-case would be to check if the resources associated to the stack are still being used or not.

  • Post deleting a stack (stackname parameter is provided) This is a place to send out a notification that a stack has been successfully or unsuccessfully deleted. Or another application could be to see if all the resources where cleaned up properly. Or to perform any more cleanup which the CloudFormation could not do.

The following repository contains the SDK that the plugins provide for a Go based plugin system:

SDK for Go based plugins.

Multiple languages

Since it's GRPC the language in which the plugin is provided is whatever the plugin's writer chooses and is supported by Furnace.

Currently three main languages are supported to write plugins in:

  • Python
  • Ruby
  • Go

Slack Plugin in Go

package main

import (
	"log"

	fplugins "github.com/go-furnace/go-furnace/furnace-aws/plugins"
	"github.com/go-furnace/sdk"
	"github.com/hashicorp/go-plugin"
)

// SlackPreCreate is an actual implementation of the furnace PreCreate plugin
// interface.
type SlackPreCreate struct{}

// Execute is the entry point to this plugin.
func (SlackPreCreate) Execute(stackname string) bool {
	api := slack.New("YOUR_TOKEN_HERE")
	params := slack.PostMessageParameters{}
	channelID, timestamp, err := api.PostMessage("#general", fmt.Sprintf("Stack with name '%s' is Done.", stackname), params)
	if err != nil {
		fmt.Printf("%s\n", err)
		return
	}
	fmt.Printf("Message successfully sent to channel %s at %s", channelID, timestamp)
	return true
}

func main() {
	plugin.Serve(&plugin.ServeConfig{
		HandshakeConfig: fplugins.Handshake,
		Plugins: map[string]plugin.Plugin{
			"slack-furnace-precreate": &sdk.PreCreateGRPCPlugin{Impl: &SlackPreCreate{}},
		},

		// A non-nil value here enables gRPC serving for this plugin...
		GRPCServer: plugin.DefaultGRPCServer,
	})
}

Sample plugin in Python

For this to work the author has to implement the proto file. A sample repository can be found here: Example for a Python Plugin.

For brevity here is the full Python source:

from concurrent import futures
import sys
import time

import grpc

import furnace_pb2
import furnace_pb2_grpc

from grpc_health.v1.health import HealthServicer
from grpc_health.v1 import health_pb2, health_pb2_grpc

class PreCreatePluginServicer(furnace_pb2_grpc.PreCreateServicer):
    """Implementation of PreCreatePlugin service."""

    def Execute(self, request, context):
        result = furnace_pb2.Proceed()
        result.failed = True

        return result

def serve():
    # We need to build a health service to work with go-plugin
    health = HealthServicer()
    health.set("plugin", health_pb2.HealthCheckResponse.ServingStatus.Value('SERVING'))

    # Start the server.
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    furnace_pb2_grpc.add_PreCreateServicer_to_server(PreCreatePluginServicer(), server)
    health_pb2_grpc.add_HealthServicer_to_server(health, server)
    server.add_insecure_port('127.0.0.1:1234')
    server.start()

    # Output information
    print("1|1|tcp|127.0.0.1:1234|grpc")
    sys.stdout.flush()

    try:
        while True:
            time.sleep(60 * 60 * 24)
    except KeyboardInterrupt:
        server.stop(0)

if __name__ == '__main__':
    serve()

The serve method here is a go-plugin requirement. To read up on it, please check-out go-plugin by HashiCorp.

Usage

After a plugin has been written simply build ( in case of Go ) or copy ( in case of Python ) it to the right location.

Furnace autodiscovers these files based on their name and loads them in order. Once that happens it will run them together at the correct event.

The following filenames should be used for the following events:

  • PreCreate: *-furnace-precreate*
  • PostCreate: *-furnace-postcreate*
  • PreDelete: *-furnace-predelete*
  • PostDelete: *-furnace-postdelete*

Separate binaries

In order to try and minimize the binary size of furnace, it has separate binaries for each service it provides.

You can find furnace-aws under aws and furnace-gcp under gcp. This way, if you plan on using only aws you don't need to worry about dependencies for Google, and vica-versa.

Contributions

Contributions are very welcomed, ideas, questions, remarks, please don't hesitate to submit a ticket. On what to do, please take a look at the ROADMAP.md file or under the Issues tab.

Pre-Binaries

Are now available under release artifacts and are automatically built by CircleCI whenever a new tag is created.

Owner
Furnace
A highly configurable Go based tool to deploy applications to cloud environments quickly and easily.
Furnace
Comments
  • Adding Google Cloud Support

    Adding Google Cloud Support

    This is a massive PR which will add sub commands and Google Cloud support to furnace.

    Further more, it will introduce a bunch of refactors and some moving around of code fragments.

  • Tags should be applied automatically.

    Tags should be applied automatically.

    Currently, the tags for instances are applied by the template. Furnace should apply its tag automatically instead of relying on the template which makes the deployment not work.

  • Consider switching to yaml based configuration.

    Consider switching to yaml based configuration.

    Instead of having everything in environment properties, consider having a yaml file as a configuration entity. Because the environment is getting crowded with various settings. It's getting hard to follow.

  • [IMPORTANT] Allow CodePush to stacks without autoscaling groups

    [IMPORTANT] Allow CodePush to stacks without autoscaling groups

    Code deployment ATM cannot be done to a single EC2 instance with no AutoScaling group.

    Creating an Autoscaling group costs nothing, but it still could be an inconvenience.

    I should allow for deploys with no autoscaling groups.

  • Try using generics for the DO implementation

    Try using generics for the DO implementation

    Now that go 1.18 provides generics, I should try and simplify a lot of the code around the DO stuff that tries to be clever with how I'm parsing the config yaml file.

Linux provisioning scripts + application deployment tools. Suitable for self-hosting and hobby-scale application deployments.

Apollo Linux provisioning scripts + application deployment tools. Suitable for self-hosting and hobby-scale application deployments. Philosophy Linux-

Feb 7, 2022
provide api for cloud service like aliyun, aws, google cloud, tencent cloud, huawei cloud and so on

cloud-fitter 云适配 Communicate with public and private clouds conveniently by a set of apis. 用一套接口,便捷地访问各类公有云和私有云 对接计划 内部筹备中,后续开放,有需求欢迎联系。 开发者社区 开发者社区文档

Dec 20, 2022
This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances

Session Manager Plugin This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances. Sess

Dec 28, 2022
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Dec 8, 2022
Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines.
Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines.

Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines, to enable infrastructure engineering teams to write tests that validate behaviour.

Sep 5, 2022
Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

Nov 10, 2021
Aws asg updater - Update AMIs within AWS Auto Scaling groups automatically.

AWS Auto Scaling Groups Updater AWS Auto Scaling group is a great way of managing Amazon EC2 instances. AWS Auto Scaling group watches the correspondi

Oct 7, 2022
Aws-secretsmanager-caching-extension - Cache server for AWS Secrets Manager
Aws-secretsmanager-caching-extension - Cache server for AWS Secrets Manager

AWS Lambda Extension / Sidecar Container Cache Server The cache server is writte

Aug 12, 2022
Pulumi-aws-iam - Reusable IAM modules for AWS

xyz Pulumi Component Provider (Go) This repo is a boilerplate showing how to cre

Jan 11, 2022
A Terraform module to manage cluster authentication (aws-auth) for an Elastic Kubernetes (EKS) cluster on AWS.

Archive Notice The terraform-aws-modules/eks/aws v.18.20.0 release has brought back support aws-auth configmap! For this reason, I highly encourage us

Dec 4, 2022
An high performance and ops-free local storage solution for Kubernetes.
An high performance and ops-free local storage solution for Kubernetes.

Carina carina 是一个CSI插件,在Kubernetes集群中提供本地存储持久卷 项目状态:开发测试中 CSI Version: 1.3.0 Carina architecture 支持的环境 Kubernetes:1.20 1.19 1.18 Node OS:Linux Filesys

May 18, 2022
Prevent Kubernetes misconfigurations from ever making it (again 😤) to production! The CLI integration provides policy enforcement solution to run automatic checks for rule violations. Docs: https://hub.datree.io
Prevent Kubernetes misconfigurations from ever making it  (again 😤) to production! The CLI integration provides policy enforcement solution to run automatic checks for rule violations.  Docs: https://hub.datree.io

What is Datree? Datree helps to prevent Kubernetes misconfigurations from ever making it to production. The CLI integration can be used locally or in

Jan 1, 2023
ControllerMesh is a solution that helps developers manage their controllers/operators better.
ControllerMesh is a solution that helps developers manage their controllers/operators better.

ControllerMesh ControllerMesh is a solution that helps developers manage their controllers/operators better. Key Features Canary update: the controlle

Jan 6, 2023
Dependency management solution for Hashicorp Terraform modules

TERRADEP This is the module dependency solution for implementing terraform's modules dependency. Using this, users can now manage dependencies both fr

Dec 21, 2021
StoneWork is a high-performance, all-(CNFs)-in-one network solution.

StoneWork, high-performance dataplane, modular control-plane solution StoneWork is used by PANTHEON.tech to integrate its CNFs on top of a single shar

Dec 23, 2022
WaffleSyrup - Simple backup solution written by Go.

WaffleSyrup Simple backup solution written by Go. Usage WaffleSyrup runs in the current working directory. It will create ./tmp directory to save tarb

Apr 22, 2022
Stream, Mutate and Sign Images with AWS Lambda and ECR
Stream, Mutate and Sign Images with AWS Lambda and ECR

ocistow About How it works Try it yourself Prerequisites CLI (cmd/ocistow) Lambda (cmd/ocistow-lambda) Deploy Invoke Verify signatures with =cosign= I

May 12, 2022
Small helper to bridge between Vault and AWS Credential Process.

vault-aws-credential-helper The Vault AWS Credential Helper is a component that can be injected into a task environment and be used as a credential he

Nov 21, 2021