Write cloud-agnostic config deployed across multiple clouds



Multy is the easiest way to deploy multi cloud infrastructure

Write cloud-agnostic config deployed across multiple clouds.

Let's try to deploy a simple virtual machine into AWS and Azure.

config {
  location = "ireland"
  clouds   = ["aws", "azure"]
}
multy "virtual_network" "example_vn" {
  name       = "example_vn"
  cidr_block = "10.0.0.0/16"
}
multy "subnet" "subnet" {
  name               = "subnet"
  cidr_block         = "10.0.2.0/24"
  virtual_network_id = example_vn.id
}
multy "virtual_machine" "vm" {
  name      = "test-vm"
  os        = "linux"
  size      = "micro"
  subnet_id = subnet.id
}

If we were to deploy this using Terraform, we would first need to understand how services such as aws_vpc and azurerm_virtual_network behave and how they differ. Then we would need to define the same infrastructure configuration twice, one for AWS and another for Azure.

This is the equivalent terraform configuration

// multy:     19 lines
// terraform: 132 lines
resource "aws_vpc" "example_vn_aws" {
  tags =  {
    Name = "example_vn"
  }

  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
}
resource "aws_internet_gateway" "example_vn_aws" {
  tags =  {
    Name = "example_vn"
  }

  vpc_id = aws_vpc.example_vn_aws.id
}
resource "aws_default_security_group" "example_vn_aws" {
  tags =  {
    Name = "example_vn"
  }

  vpc_id = aws_vpc.example_vn_aws.id

  ingress {
    protocol    = "-1"
    from_port   = 0
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
    self        = true
  }

  egress {
    protocol    = "-1"
    from_port   = 0
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
    self        = true
  }
}
resource "aws_subnet" "subnet_aws" {
  tags =  {
    Name = "subnet"
  }

  cidr_block = "10.0.2.0/24"
  vpc_id     = aws_vpc.example_vn_aws.id
}
resource "aws_instance" "vm_aws" {
  tags =  {
    Name = "test-vm"
  }

  ami           = "ami-09d4a659cdd8677be"
  instance_type = "t2.nano"
  subnet_id     = aws_subnet.subnet_aws.id
}
resource "azurerm_virtual_network" "example_vn_azure" {
  resource_group_name = azurerm_resource_group.vn-rg.name
  name                = "example_vn"
  location            = "northeurope"
  address_space       = ["10.0.0.0/16"]
}
resource "azurerm_route_table" "example_vn_azure" {
  resource_group_name = azurerm_resource_group.vn-rg.name
  name                = "example_vn"
  location            = "northeurope"

  route {
    name           = "local"
    address_prefix = "0.0.0.0/0"
    next_hop_type  = "VnetLocal"
  }
}
resource "azurerm_subnet" "subnet_azure" {
  resource_group_name  = azurerm_resource_group.vn-rg.name
  name                 = "subnet"
  address_prefixes     = ["10.0.2.0/24"]
  virtual_network_name = azurerm_virtual_network.example_vn_azure.name
}
resource "azurerm_subnet_route_table_association" "subnet_azure" {
  subnet_id      = azurerm_subnet.subnet_azure.id
  route_table_id = azurerm_route_table.example_vn_azure.id
}
resource "azurerm_network_interface" "vm_azure" {
  resource_group_name = azurerm_resource_group.vm-rg.name
  name                = "test-vm"
  location            = "northeurope"

  ip_configuration {
    name                          = "internal"
    private_ip_address_allocation = "Dynamic"
    subnet_id                     = azurerm_subnet.subnet_azure.id
    primary                       = true
  }
}
resource "azurerm_linux_virtual_machine" "vm_azure" {
  resource_group_name   = azurerm_resource_group.vm-rg.name
  name                  = "test-vm"
  location              = "northeurope"
  size                  = "Standard_B1ls"
  network_interface_ids = [azurerm_network_interface.vm_azure.id]

  os_disk {
    caching              = "None"
    storage_account_type = "Standard_LRS"
  }

  admin_username = "multyadmin"
  admin_password = "Multyadmin090#"

  source_image_reference {
    publisher = "OpenLogic"
    offer     = "CentOS"
    sku       = "7_9-gen2"
    version   = "latest"
  }

  disable_password_authentication = false
}
resource "azurerm_resource_group" "vm-rg" {
  name     = "vm-rg"
  location = "northeurope"
}
resource "azurerm_resource_group" "vn-rg" {
  name     = "vn-rg"
  location = "northeurope"
}
provider "aws" {
  region = "eu-west-1"
}
provider "azurerm" {
  features {}
}

With Multy, you write once, and deploy anywhere.

Getting started


  • Docs: check our documentation page to get started
  • Examples: look over some common examples
  • Resources: currently supported resources
Comments
  • feat: configurable DB port

    feat: configurable DB port

    Added configurable DB port as mentioned in #198 but unsure about the tests, some are failing. Also wasn't sure about adding the port to the Azure MySQL database type since that isn't configurable currently

  • Implement SubnetFromState

    Implement SubnetFromState

    Read attributes from internal terraform state and convert to object as per https://github.com/multycloud/multy/blob/main/resources/types/virtual_network.go#L65

  • Add overrides for VM's images

    Add overrides for VM's images

    Multy provides a cloud-agnostic way to specify an image. However, sometimes users want to use a specific AWS ami, or equivalent in other clouds.

    This issue adds:

    1. Override field to VM's overrides
    • https://github.com/multycloud/multy/blob/main/api/proto/resourcespb/virtual_machine.proto#L41 - string that will map to aws ami
    • https://github.com/multycloud/multy/blob/main/api/proto/resourcespb/virtual_machine.proto#L45 - message that will map to azure img reference block
    • https://github.com/multycloud/multy/blob/main/api/proto/resourcespb/virtual_machine.proto#L49 - string that will map to gcp iamge
    1. Use overrides if specified in translation layer. Similar to how it's done currently for vm sizes - see example for AWS

    2. Add tests for each cloud using their overrides. Copy from this example - https://github.com/multycloud/multy/tree/main/test/_configs/virtual_machine/virtual_machine_size_override but override the image instead of the size. You can see what's currently being generated by running go test ./test --write_generated - it will create a generated.tf file with the current output of the translation

  • Add mapping for GCP virtual machine sizes

    Add mapping for GCP virtual machine sizes

    We only have mapping for general purpose compute, we're missing compute optimized and memory optimized for GCP.

    Mapping is defined in https://github.com/multycloud/multy/blob/main/resources/common/vm_size.go

  • Validate network security group resource

    Validate network security group resource

    • Check that name matches azure's restrictions: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules
    • Check that security rules have a valid port range
    • Check that security rules cidr block is valid
    • Check that security rules protocl is valid (check possible values https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/network_security_rule)
    • Check that priority is between 100 and 4096
  • Validate virtual_network resource

    Validate virtual_network resource

    • Check that the CIDR block is valid
    • Check that the name matches the restrictions in https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules

    Matching issue in terraform provider: https://github.com/multycloud/terraform-provider-multy/issues/54

  • feat: Implemented RouteTableFromState

    feat: Implemented RouteTableFromState

    Implemented RouteTableFromState as mentioned in Issue #214

    Unsure about setting the routes on AWS and Azure, and didn't set the VirtualNetworkId for Azure since the Terraform docs for Azure Route Table don't show it tied to a Virtual Network https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/route_table

    Not sure how to test locally, so feedback is greatly appreciated!

  • Implement RouteTableFromState

    Implement RouteTableFromState

    Read attributes from internal terraform state and convert to object as per https://github.com/multycloud/multy/blob/main/resources/types/virtual_network.go#L65

  • Add MariaDB and PostgreSQL to database engines

    Add MariaDB and PostgreSQL to database engines

    • Add engine to proto definition in https://github.com/multycloud/multy/blob/main/api/proto/resourcespb/database.proto
    • Translate it to AWS and Azure in https://github.com/multycloud/multy/blob/main/resources/types/database.go
    • Add tests to tests/_configs/database
  • Add kubernetes output to be used for kubectl config

    Add kubernetes output to be used for kubectl config

    Aws uses a token based system (tf has eks_cluster_auth data source for this) while azure uses a username+password and an asymmetric key authentication system

  • Web app demo problems

    Web app demo problems

    • If db is azure, it doesn't work
    • Aws runs the custom data script in root mode but azure doesn't (maybe we just need to make sure both clouds use the same vm image?)
  • Add overrides for locations

    Add overrides for locations

    Multy has a mapping for regions https://github.com/multycloud/multy/blob/4f57fcf3826b13653b9920d6af76bbca2cd773ba/resources/common/cloud_provider.go#L16. In some cases, users might want a different region that is not covered by us.

    To do to complete this issue:

    • Add override field to https://github.com/multycloud/multy/blob/main/api/proto/commonpb/common.proto#L79
    • Use override if specified in https://github.com/multycloud/multy/blob/main/resources/resource_with_id.go#L39 (and check for any other places where it might be accessed directly)
    • Add tests - probably something simple like similar to this https://github.com/multycloud/multy/tree/main/test/_configs/virtual_network/virtual_network or also some unit tests
  • Add managed distributed cache resource

    Add managed distributed cache resource

    Distributed cache is offered among AWS, Azure and GCP.

    This issue tracks the first implementation of a managed distributed cache resource, supporting Redis engine (memcached in the future), and both stand-alone and replicated servers.

    GCP doesn't have Redis cluster support and Azure doesn't allow you to specify the version (which is 4.0 for now).

    AWS: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_cluster Azure: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/redis_cache GCP: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/redis_instance

  • Add E2E test for virtual network and subnet

    Add E2E test for virtual network and subnet

    Following assumptions need to be tested: By default, 2 resources in a virtual network can communicate with each other on any port, without internet access. By default, a resource placed in a virtual network does not have access to the internet. Assigned IPs are within the specified subnet CIDR block range.

  • Support highly available databases

    Support highly available databases

    • AWS supports multi-az databases by specifying a parameter (multi_az).
    • Azure, however, has a new set of database instances called "flexible servers" and that's the recommended way to deploy databases today. Those also allow zone redundant deployments. Not available for mariadb apparently.
    • GCP also has a parameter (availability_type) for multi-az deployments
  • Storage lifecycle rules

    Storage lifecycle rules

    https://docs.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html https://cloud.google.com/storage/docs/lifecycle

A collection of cloud security icons :cloud::lock:
A collection of cloud security icons :cloud::lock:

Cloud Security Icons These icons are published under the extremely permissive Creative Commons Zero v1.0 Universal license. Downloads We provide all i

Jan 7, 2023
golang sdk for BRCC ( BRCC:Better Remote Config Center)
golang sdk for BRCC ( BRCC:Better Remote Config Center)

brcc-go-sdk golang sdk for BRCC BRCC:Better Remote Config Center

Dec 6, 2021
GoStorm is a Go library that implements the communications protocol required to write Storm spouts and Bolts in Go that communicate with the Storm shells.

gostorm godocs GoStorm is a Go library that implements the communications protocol required for non-Java languages to communicate as part of a storm t

Sep 27, 2022
Wechat Pay SDK(V3) Write by Go.

WechatPay GO(v3) Introduction Wechat Pay SDK(V3) Write by Go. API V3 of Office document is here. Features Signature/Verify messages Encrypt/Decrypt ce

May 23, 2022
Elastos.ELA.Rosetta.API - How to write a Rosetta server and use either the Client package or Fetcher package to communicate

Examples This folder demonstrates how to write a Rosetta server and how to use e

Jan 17, 2022
Prismplus - Prism+ lets you multicast your rtmp stream to multiple destinations
Prismplus - Prism+ lets you multicast your rtmp stream to multiple destinations

prism+ Use at your own risk! It has worked for us.. but very much alpha quality!

Nov 9, 2022
Firebase Cloud Messaging for application servers implemented using the Go programming language.

Firebase Cloud Notifications Client Firebase Cloud Messaging for application servers implemented using the Go programming language. It's designed for

Dec 17, 2022
Google Cloud Messaging for application servers implemented using the Go programming language.

gcm The Android SDK provides a nice convenience library (com.google.android.gcm.server) that greatly simplifies the interaction between Java-based app

Sep 27, 2022
Google Cloud Client Libraries for Go.

Google Cloud Client Libraries for Go Go packages for Google Cloud Platform services. import "cloud.google.com/go" To install the packages on your syst

Jan 1, 2023
Cloud governance reports from native services in a clear and readable digest
Cloud governance reports from native services in a clear and readable digest

cloudig, or Cloudigest, is a simple CLI tool for creating reports from various cloud sources with user-provided comments. It is written in Go and curr

Nov 10, 2022
Abusing Discord for unlimited cloud storage

Discord Cloud Storage Abusing Discord's servers for unlimited cloud storage! So, what is this? Infamous 8MB limit for non-nitro users can get pretty a

Nov 26, 2022
Pulumi - Modern Infrastructure as Code. Any cloud, any language 🚀
Pulumi - Modern Infrastructure as Code. Any cloud, any language 🚀

Pulumi's Infrastructure as Code SDK is the easiest way to create and deploy cloud software that use containers, serverless functions, hosted services,

Dec 30, 2022
Go server SDK for IBM Cloud Event Notifications service

IBM Cloud Event Notifications Go Admin SDK Go client library to interact with the various IBM Cloud Event Notifications APIs. Disclaimer: this SDK is

Dec 14, 2022
Alibaba Cloud foasconsole SDK for Go

English | 简体中文 Alibaba Cloud foasconsole SDK for Go Requirements It's necessary for you to make sure your system have installed Go environment which v

Nov 1, 2021
Alibaba Cloud RMC SDK for Go

English | 简体中文 Alibaba Cloud RMC SDK for Go Requirements It's necessary for you to make sure your system have installed Go environment which version g

Nov 5, 2021
Alibaba Cloud BatchCompute SDK for Go

English | 简体中文 Alibaba Cloud BatchCompute SDK for Go Requirements It's necessary for you to make sure your system have installed Go environment which

Nov 15, 2021
Alibaba Cloud GEMP SDK for Go

English | 简体中文 Alibaba Cloud GEMP SDK for Go Requirements It's necessary for you to make sure your system have installed Go environment which version

Nov 16, 2021
Alibaba Cloud PTS SDK for Go
Alibaba Cloud PTS SDK for Go

Alibaba Cloud PTS SDK for Go

Dec 27, 2021
Alibaba Cloud xixikf SDK for Go

English | 简体中文 Alibaba Cloud xixikf SDK for Go Requirements It's necessary for you to make sure your system have installed Go environment which versio

Nov 25, 2021