Framework for building distributed services with Web Assembly

Tarmac

Framework for building distributed services with Web Assembly

PkgGoDev Go Report Card Documentation

Tarmac is a unique framework designed for the next generation of distributed systems. At its core, like many other microservice frameworks, Tarmac is focused on abstracting the complexities of building cloud-native services allowing users to focus more on business logic and less on boilerplate code.

What makes Tarmac unique is that, unlike most microservice frameworks, Tarmac is language agnostic. Using Web Assembly (WASM), Tarmac users can write their business logic in many different languages such as Rust, Go, Javascript, or even Swift; and run it all using the same core framework.

Tarmac vs. Serverless Functions

Tarmac shares many traits with Serverless Functions and Functions as a Service (FaaS) platforms. Tarmac makes it easy for developers to deploy functions and microservices without writing repetitive boilerplate code. As a developer, you can create a production-ready service in less than 100 lines of code.

But Tarmac takes Serverless Functions further. In general, FaaS platforms provide a simple runtime for user code. If a function requires any dependency (i.e., a Database), the developer-provided function code must maintain the database connectivity and query calls.

Using the power of Web Assembly, Tarmac not only provides functions a secure sandboxed runtime environment, but it also provides abstractions that developers can use to interact with platform capabilities such as Databases, Caching, Metrics, and even Dynamic Configuration.

In many ways, Tarmac is more akin to a microservices framework with the developer experience of a FaaS platform.

Quick Start

At the moment, Tramac is executing WASM functions by executing a defined set of function signatures. When Tarmac receives an HTTP GET request, it will call the function's registered under the http:GET signature.

As part of the WASM Function, users must register their handlers using the pre-defined function signatures.

To understand this better, look at one of our simple examples written in Rust (found in example/).

// Tac is a small, simple Rust program that is an example WASM function for Tarmac.
// This program will accept a Tarmac server request, log it, and echo back the payload
// but with the payload reversed.
extern crate wapc_guest as guest;
extern crate base64;
use serde::{Deserialize, Serialize};
use serde_json;
use std::collections::HashMap;
use guest::prelude::*;

#[derive(Serialize, Deserialize)]
struct ServerRequest {
  headers: HashMap<String, String>,
  payload: String,
}

#[derive(Serialize, Deserialize)]
struct ServerResponse {
  headers: HashMap<String, String>,
  status: Status,
  payload: String,
}

#[derive(Serialize, Deserialize)]
struct Status {
  code: u32,
  status: String,
}

fn main() {}

#[no_mangle]
pub extern "C" fn wapc_init() {
  // Add Handler for the GET request
  register_function("http:GET", fail_handler);
  // Add Handler for the POST request
  register_function("http:POST", handler);
  // Add Handler for the PUT request
  register_function("http:PUT", handler);
  // Add Handler for the DELETE request
  register_function("http:DELETE", fail_handler);
}

// fail_handler will accept the server request and return a server response
// which rejects the client request
fn fail_handler(_msg: &[u8]) -> CallResult {
  // Create the response
  let rsp = ServerResponse {
      status: Status {
        code: 503,
        status: "Not Implemented".to_string(),
      },
      payload: "".to_string(),
      headers: HashMap::new(),
  };

  // Marshal the response
  let r = serde_json::to_vec(&rsp).unwrap();

  // Return JSON byte array
  Ok(r)
}

// handler is a simple example of a Tarmac WASM function written in Rust.
// This function will accept the server request, log it, and echo back the payload
// but with the payload reversed.
fn handler(msg: &[u8]) -> CallResult {
  // Perform a host callback to log the incoming request
  let _res = host_call("tarmac", "logger", "debug", &msg.to_vec());

  // Unmarshal the request
  let rq: ServerRequest = serde_json::from_slice(msg).unwrap();

  // Decode Payload
  let b = base64::decode(rq.payload).unwrap();
  // Convert to a String
  let s = String::from_utf8(b).expect("Found Invalid UTF-8");
  // Reverse it and re-encode
  let enc = base64::encode(s.chars().rev().collect::<String>());

  // Create the response
  let rsp = ServerResponse {
      status: Status {
        code: 200,
        status: "OK".to_string(),
      },
      payload: enc,
      headers: HashMap::new(),
  };

  // Marshal the response
  let r = serde_json::to_vec(&rsp).unwrap();

  // Return JSON byte array
  Ok(r)
}

Tarmac passes the HTTP Context and Payload to the WASM function via the incoming msg. The msg is a JSON that contains Headers and a Payload which is Base64 encoded but otherwise untouched.

To compile the example above, run:

$ cd example/tac/rust
$ make build

Once compiled, users can run Tarmac via Docker using the following command:

$ docker run -p 8080:8080 \
  -e "APP_ENABLE_TLS=false" -e "APP_LISTEN_ADDR=0.0.0.0:8080" \
  -v ./functions:/functions madflojo/tarmac

With Tarmac now running, we can access our WASM function using any HTTP Client such as curl.

$ curl -v --data "Tarmac Example" http://localhost:8080
Owner
Benjamin Cane
Thoughts and opinions are my own.
Benjamin Cane
Comments
  • WASI Module not instantiated error when running example

    WASI Module not instantiated error when running example

    Running on WSL2 Ubuntu Docker version 20.10.14, build a224086

    The source was cloned and the example build instructions were followed

    $ cd example/tac/go
    $ make build
    make build
    mkdir -p functions
    ## Run TinyGo build via Docker because waPC requires a <0.18.0 version of TinyGo to work right
    docker run -v `pwd`/:/build -w /build tinygo/tinygo:0.17.0 tinygo build -o /build/functions/tarmac.wasm -target wasi /build/main.go
    go: downloading github.com/wapc/wapc-guest-tinygo v0.3.0
    

    The tarmac wasm file is generated after running the build command:

    $ ls -l functions/
    total 220
    -rwxr-xr-x 1 root root 221650 Jul 18 11:21 tarmac.wasm
    

    Running the tarmac container:

    docker run -p 8080:8080  \
        -e "APP_ENABLE_TLS=false" \
        -e "APP_LISTEN_ADDR=0.0.0.0:8080" \
        -v $(pwd)/functions:/functions madflojo/tarmac
    
    time="2022-07-18T15:00:33Z" level=warning msg="No Config file found, loaded config from Environment - Default path ./conf"
    time="2022-07-18T15:00:33Z" level=info msg="KV Store not configured, skipping"
    time="2022-07-18T15:00:33Z" level=info msg="SQL DB not configured, skipping"
    time="2022-07-18T15:00:33Z" level=fatal msg="Service stopped - unable to create module pool for wasm file /functions/tarmac.wasm - module[wasi_unstable] not instantiated"
    
  • deps: updates wazero to 1.0.0-pre.1

    deps: updates wazero to 1.0.0-pre.1

    This updates wazero to 1.0.0-pre.1

    A new pre-release will happen at least once each month until 1.0 in February 2023.

    Note: Release notes will be posted in the next day or two.

    Meanwhile, we've also opened a gophers slack #wazero channel for support, updates and conversation! Note: You may need an invite to join gophers.

  •  GGO free wasm runner

    GGO free wasm runner

    Wazero is faster than the others due to less overhead

    https://wazero.io/

    It can run within envoy or alone Config can be file based or off envoy.

    It can also run wasi and wasm compiled golang .

  • deps: updates wazero to 1.0.0-pre.4

    deps: updates wazero to 1.0.0-pre.4

    This updates wazero to 1.0.0-pre.4.

    Notably, v1.0.0-pre.4:

    • improves module initialization speed
    • supports listeners in the compiler engine
    • supports WASI fd_pread, fd_readdir and path_filestat_get
  • Updating wapc to use wazero

    Updating wapc to use wazero

    This PR closes #30 and updates the wapc-go package to use a C-free WASM runtime engine https://github.com/tetratelabs/wazero

    One Caveat for this PR is that WASM modules will need to be compiled with a higher version of TinyGo than used in the past. I.E. TinyGo 0.24.

  • Adding m-TLS for Authentication

    Adding m-TLS for Authentication

    After looking at #34 I started looking at the different Auth N/Z options available. I will be adding JWT-based authentication later but I wanted to get started with m-TLS which is pretty straightforward.

  • Fixing WASI Module error

    Fixing WASI Module error

    When looking more into the error and reading some of the maintainers for wapc comments on previous Pull Requests, I came across the release notes for this: https://github.com/wapc/wapc-guest-tinygo/releases/tag/v0.3.1

    There were some changes in TinyGo v0.19 that the older version of wapc-guest-tinygo didn't work well with. But with the new version v0.3.1, everything seems to be working fine.

  • Changing to TinyGo v18

    Changing to TinyGo v18

    As reported in #35, after updating wapc and using wazero POST requests fail for all TinyGo versions greater than 0.17 except for 0.18. So, for now, I am updating the Makefiles to use v0.18.

  • Revamping function inputs

    Revamping function inputs

    This pull request updates quite a bit of how Tarmac works. The goal is to make it a little more default and simple to configure/use. Expect some more changes like this as it goes.

  • Toolkit

    Toolkit

    This pull request is mainly a refactor in that I'm moving most of the callback functionality into their packages. The cool thing about this is, it starts opening the door for users to use Tarmac as a toolkit to add functionality to their services running WASM.

    Later I'll add some examples and documentation of just that.

  • HTTP Callback

    HTTP Callback

    This Pull Request adds an HTTP Client Callback that allows WASM functions to execute HTTP calls. This is loosely molded after how Envoy exposes an HTTP Call function to WASM plugins.

  • Add tags to GitHub Actions

    Add tags to GitHub Actions

    https://github.com/madflojo/tarmac/blob/7f6349796c6612c8d45f66edbeb4c66d70752798/.github/workflows/docker.yml#L3-L5

    Currently, release tags don't have automatic docker publishing, tests, or any other actions being performed.

  • Auth - authz

    Auth - authz

    As far as I can see there is not any auth or authz in tarmac.

    i generally use NATS for rerouting events into tarmac . Nats just wants a jwt to control Auth and Authz. But even without NATS , Tarnac needs to assert who can do what.

    https://github.com/pocketbase/pocketbase looks like a nice solution to this. .Or at least to use as one way to add auth / authz. It’s probable that others might want a different solution and that’s why jwt is loose coupled.

    the cool thing about pocketbaae is that it’s real time and simple.

    have a look

  • embed

    embed

    tinygo now supports embed package which might be useful for functions that need resources.

    See: https://github.com/tinygo-org/tinygo/releases/tag/v0.24.0

  • More Examples

    More Examples

    Tarmac is still pretty new, and while I know Go pretty well Rust and AssemblyScript are still fairly new to me. It would be great to have some more examples created for Tarmac.

    https://github.com/madflojo/tarmac/tree/main/example

A package to build progressive web apps with Go programming language and WebAssembly.
A package to build progressive web apps with Go programming language and WebAssembly.

Go-app is a package for building progressive web apps (PWA) with the Go programming language (Golang) and WebAssembly (Wasm). Shaping a UI is done by

Dec 30, 2022
A package to build progressive web apps with Go programming language and WebAssembly.
A package to build progressive web apps with Go programming language and WebAssembly.

Go-app is a package for building progressive web apps (PWA) with the Go programming language (Golang) and WebAssembly (Wasm). Shaping a UI is done by

Jan 2, 2023
Bed and Breakfast web app written in Go

BOOKINGS AND RESERVATIONS This repository contains the files for my RareBnB application RareBnB is an "AirBnB" style clone providing a user the abilit

Jan 11, 2022
Dom - A Go API for different Web APIs for WebAssembly target

Go DOM binding (and more) for WebAssembly This library provides a Go API for dif

Jan 7, 2023
WebAssembly Lightweight Javascript Framework in Go (AngularJS Inspired)

Tango Lightweight WASM HTML / Javascript Framework Intro WebAssembly is nice, Go on the web is nice, so I ported Tangu to Go and WebAssembly. Tangu is

Dec 20, 2022
The Bhojpur Ara is a software product used for automated resource assembly within Bhojpur.NET Platform ecosystem to enable delivery of applications and services.

Bhojpur Ara - Automated Resource Assembly The Bhojpur Ara is a service product used for automated resource assembly within the Bhojpur.NET Platform ec

Apr 28, 2022
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Jun 1, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Dec 9, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Dec 9, 2022
Mortar is a GO framework/library for building gRPC (and REST) web services.
Mortar is a GO framework/library for building gRPC (and REST) web services.

Mortar Mortar is a GO framework/library for building gRPC (and REST) web services. Mortar has out-of-the-box support for configuration, application me

Dec 26, 2022
Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

Nov 10, 2021
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7天用Go动手写/从零实现系列

7 days golang programs from scratch README 中文版本 7天用Go从零实现系列 7天能写什么呢?类似 gin 的 web 框架?类似 groupcache 的分布式缓存?或者一个简单的 Python 解释器?希望这个仓库能给你答案

Jan 5, 2023
This library contains utilities that are useful for building distributed services.

Grafana Dskit This library contains utilities that are useful for building distributed services. Current state This library is still in development. D

Jan 2, 2023
Go Lang Web Assembly bindings for DOM, HTML etc

WebAPI Go Language Web Assembly bindings for DOM, HTML etc WARNING: The current API is in very early state and should be consider to be expremental. T

Dec 28, 2022
package for building REST-style Web Services using Go

go-restful package for building REST-style Web Services using Google Go Code examples using v3 REST asks developers to use HTTP methods explicitly and

Jan 1, 2023
package for building REST-style Web Services using Go

go-restful package for building REST-style Web Services using Google Go Code examples using v3 REST asks developers to use HTTP methods explicitly and

Jan 1, 2023
High-performance framework for building redis-protocol compatible TCP servers/services

Redeo The high-performance Swiss Army Knife for building redis-protocol compatible servers/services. Parts This repository is organised into multiple

Jan 4, 2023
⚡️ A Go framework for rapidly building powerful graphql services

Thunder is a Go framework for rapidly building powerful graphql servers. Thunder has support for schemas automatically generated from Go types, live q

Dec 24, 2022
Distributed Commit Log from Travis Jeffery's Distributed Services book

proglog A distributed commit log. This repository follows the book "Distributed Services with Go" by Travis Jeffrey. The official repository for this

May 23, 2022
Golang evasion tool, execute-assembly .Net file

?? Frog For Automatic Scan ?? Doge For Defense Evasion&Offensive Security Doge-Assembly Golang evasion tool, execute-assembly .Net file Intro Are you

Jan 8, 2023