This application demonstrates how to launch high-performance "serverless" functions from the YoMo framework to process streaming data. The functions are embedded in a WebAssembly VM, WasmEdge, for safety, security, portability, and manageability.

Streaming Image Recognition by WebAssembly

Youtube: YoMo x WasmEdge

This project demonstrates how to process a video stream in real-time using WebAssembly and apply a pre-trained food classification model to each frame of the video in order to determine if food is present in that frame, all by integrating WasmEdge into YoMo serverless.

Open-source projects that we used:

  • Serverless stream processing framework YoMo
  • Integrate with WasmEdge to introduce WebAssembly, interop TensorFlow Lite model
  • A deep learning model found on TensorFlow Hub; make sure to download TFLite (aiy/vision/classifier/food_V1), which was created by Google

Advantages:

  • ⚡️ Low-latency: Streaming data processing applications can now be done in far edge data centers thanks to YoMo's highly efficient network services
  • 🔐 Security: WasmEdge runs the data processing function in a WebAssembly sandbox for isolation, safety, and hot deployment
  • 🚀 High Performance: Compared with popular containers, such as Docker, WasmEdge can be up to 100x faster at startup and have a much smaller footprint
  • 🎯 Edge Devices: As WasmEdge consumes much less resources than Docker, it is now possible to run data processing applications on edge devices

Steps to run

1. Clone This Repository

$ git clone https://github.com/yomorun/yomo-wasmedge-tensorflow.git

2. Install YoMo CLI

$ go install github.com/yomorun/cli/yomo@latest
$ yomo version
YoMo CLI version: v0.0.5

Or, you can download the pre-built binary tarball yomo-v0.0.5-x86_64-linux.tgz.

Details about YoMo CLI installation can be found here.

3. Install WasmEdge Dependencies

Install WasmEdge

$ wget https://github.com/WasmEdge/WasmEdge/releases/download/0.8.0/WasmEdge-0.8.0-manylinux2014_x86_64.tar.gz
$ tar -xzf WasmEdge-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo cp WasmEdge-0.8.0-Linux/include/wasmedge.h /usr/local/include
$ sudo cp WasmEdge-0.8.0-Linux/lib64/libwasmedge_c.so /usr/local/lib
$ sudo ldconfig

Or, you can build from the source.

Install WasmEdge-tensorflow

Install tensorflow dependencies for manylinux2014 platform

$ wget https://github.com/second-state/WasmEdge-tensorflow-deps/releases/download/0.8.0/WasmEdge-tensorflow-deps-TF-0.8.0-manylinux2014_x86_64.tar.gz
$ wget https://github.com/second-state/WasmEdge-tensorflow-deps/releases/download/0.8.0/WasmEdge-tensorflow-deps-TFLite-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo tar -C /usr/local/lib -xzf WasmEdge-tensorflow-deps-TF-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo tar -C /usr/local/lib -xzf WasmEdge-tensorflow-deps-TFLite-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo ln -sf libtensorflow.so.2.4.0 /usr/local/lib/libtensorflow.so.2
$ sudo ln -sf libtensorflow.so.2 /usr/local/lib/libtensorflow.so
$ sudo ln -sf libtensorflow_framework.so.2.4.0 /usr/local/lib/libtensorflow_framework.so.2
$ sudo ln -sf libtensorflow_framework.so.2 /usr/local/lib/libtensorflow_framework.so
$ sudo ldconfig

Install WasmEdge-tensorflow:

$ wget https://github.com/second-state/WasmEdge-tensorflow/releases/download/0.8.0/WasmEdge-tensorflow-0.8.0-manylinux2014_x86_64.tar.gz
$ wget https://github.com/second-state/WasmEdge-tensorflow/releases/download/0.8.0/WasmEdge-tensorflowlite-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo tar -C /usr/local/ -xzf WasmEdge-tensorflow-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo tar -C /usr/local/ -xzf WasmEdge-tensorflowlite-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo ldconfig

Install WasmEdge-image:

$ wget https://github.com/second-state/WasmEdge-image/releases/download/0.8.0/WasmEdge-image-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo tar -C /usr/local/ -xzf WasmEdge-image-0.8.0-manylinux2014_x86_64.tar.gz
$ sudo ldconfig

If you have any questions about installation, please refer to the official documentation. Currently, this project works on Linux machines only.

Install video and image processing dependencies

$ sudo apt-get update
$ sudo apt-get install -y ffmpeg libjpeg-dev libpng-dev

4. Write your Streaming Serverless function

Write app.go to integrate WasmEdge-tensorflow:

Get WasmEdge-go:

$ cd flow
$ go get -u github.com/second-state/WasmEdge-go/wasmedge

Download pre-trained TensorflowLitee model: lite-model_aiy_vision_classifier_food_V1_1.tflite, store to rust_mobilenet_foods/src:

$ wget 'https://storage.googleapis.com/tfhub-lite-models/google/lite-model/aiy/vision/classifier/food_V1/1.tflite' -O ./rust_mobilenet_food/src/lite-model_aiy_vision_classifier_food_V1_1.tflite

Compile to wasm file:

Install rustc and cargo

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ export PATH=$PATH:$HOME/.cargo/bin
$ rustc --version

Set default rust version to 1.50.0: $ rustup default 1.50.0

Install rustwasmc

$ curl https://raw.githubusercontent.com/second-state/rustwasmc/master/installer/init.sh -sSf | sh
$ cd rust_mobilenet_food
$ rustwasmc build
# The output WASM will be `pkg/rust_mobilenet_food_lib_bg.wasm`.

Copy pkg/rust_mobilenet_food_lib_bg.wasm to flow directory:

$ cp pkg/rust_mobilenet_food_lib_bg.wasm ../.

5. Run YoMo Orchestrator Server

  $ yomo serve -c ./zipper/workflow.yaml

6. Run Streaming Serverless function

$ cd flow
$ go run --tags "tensorflow image" app.go

7. Demonstrate video stream

Download this demo vide: hot-dog.mp4, store to source directory, then run:

$ wget -P source 'https://github.com/yomorun/yomo-wasmedge-tensorflow/releases/download/v0.1.0/hot-dog.mp4'
$ go run ./source/main.go ./source/hot-dog.mp4

8. Result

YoMo-WasmEdge

Owner
YoMo
Realtime IoT & edge computing framework engaging 5G
YoMo
Comments
  • Trouble in leveraging generated rust_mobilenet_food_lib_bg.so

    Trouble in leveraging generated rust_mobilenet_food_lib_bg.so

    Hello, interesting project. I encounter some problems when using it. First, I uses the rustwasmc build --enable-aot command to generate the .so file by myself. However, there arises the error When executing function name: "__wbindgen_malloc" panic: toWasmEdgeValueSlideBindgen(): malloc failed. How can I solve the problem? And how to correctly generate the wasm .so file for use? Another question is that when I use the .wasm file instead of the .so file in the function LoadWasmFile, the DNN inference time (denoted by Finished post-processing in) increases from about 200 ms to 800 ms, which seems so strange. I cannot figure it out. Thank you for the answers in advance.

  • Can this demo code works with other TFlite models from TensorFlow Hub?

    Can this demo code works with other TFlite models from TensorFlow Hub?

    I have tried to play with other modules like https://tfhub.dev/google/lite-model/aiy/vision/classifier/insects_V1/3 and https://tfhub.dev/google/aiy/vision/classifier/plants_V1/1 and I recompile the code with rustwasmc , when try to infer it always say

    RUST: Loaded image in ... 16.25169ms RUST: index -1, prob 0 RUST: Finished post-processing in ... 268.088698ms

    the modify part for lib.rs as follow

    session.add_input("input", &flat_img, &[1, 192, 192, 3])
           .add_output("MobilenetV2/Predictions/Softmax")
           .run();
    let res_vec: Vec<f32> = session.get_output("MobilenetV2/Predictions/Softmax");
    

    Seems it only works for the food module?

  • lite-model_aiy_vision_classifier_food_V1_1.tflite file not exist

    lite-model_aiy_vision_classifier_food_V1_1.tflite file not exist

    In the latest code flow/rust_mobilenet_food/src/lib.rs line 9 it try to load the model file lite-model_aiy_vision_classifier_food_V1_1.tflite but this file doens't exsit. In v0.2.0 release, the file name is rust_mobilenet_food_lib_bg.so and can be downloaded at the download page.

  • Error: CRYPTO_ERROR (0x178): tls: no application protocol in quic-Go

    Error: CRYPTO_ERROR (0x178): tls: no application protocol in quic-Go

    After solving a lot of problems, I finally come to the final step:

    1. yomo serve -c ./workflow.yaml I get:
    Using config file: ./workflow.yaml
    ℹ️   Running YoMo-Zipper...
    2021/10/27 08:22:44 [core:server] ✅ (name:Service) Listening on: 127.0.0.1:9000, QUIC: [v1 draft-29]
    2021/10/27 08:22:44 [yomo:zipper] Listening SIGUSR1, SIGUSR2, SIGTERM/SIGINT...
    
    1. go run ./source/main.go ./source/hot-dog.mp4 I get:
    Go: Args: [/tmp/go-build2709539636/b001/exe/main ./source/hot-dog.mp4]
    2021/10/27 08:22:50 Connecting to zipper localhost:9000 ...
    client [NewClient] Error: CRYPTO_ERROR (0x178): tls: no application protocol
    2021/10/27 08:22:50 ❌ Emit the data to yomo-zipper failure with err: CRYPTO_ERROR (0x178): tls: no application protocol
    
    1. go run --tags "tensorflow image" app.go I get:
    /tmp/go-build1371428046/b001/exe/app: Relink `/usr/local/lib/libtensorflow_framework.so.2' with `/lib/x86_64-linux-gnu/libz.so.1' for IFUNC symbol `crc32_z'
    2021/10/27 07:29:01 Connecting to zipper localhost:9000 ...
    client [NewClient] Error: CRYPTO_ERROR (0x178): tls: no application protocol
    2021/10/27 07:29:01 ❌ Connect to zipper failure: CRYPTO_ERROR (0x178): tls: no application protocol
    

    It's my configuration:

    yomo-cli: YoMo CLI version: v0.1.3
    WasmEdge: 0.8.2-manylinux2014_x86_64
    

    How can I solve this problem? I can't get this ERROR when I run the basic example in YoMo.

  • failed to run flow/app.go

    failed to run flow/app.go

    use WasmEdge v0.10.0 with github.com/second-state/WasmEdge-go v0.10.0,

    cd flow go run --tags "tensorflow image" app.go # github.com/second-state/wasmedge-bindgen/host/go ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:43:24: undefined: wasmedge.ImportObject ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:61:26: undefined: wasmedge.NewImportObject ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:68:7: b.vm.RegisterImport undefined (type *wasmedge.VM has no field or method RegisterImport) ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:84:28: b.vm.GetStore().FindMemory undefined (type *wasmedge.Store has no field or method FindMemory) ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:184:28: b.vm.GetStore().FindMemory undefined (type *wasmedge.Store has no field or method FindMemory) ../../../go/pkg/mod/github.com/second-state/[email protected]/host/go/bindgen.go:261:28: b.vm.GetStore().FindMemory undefined (type *wasmedge.Store has no field or method FindMemory)

High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Dec 9, 2021
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

Sep 19, 2021
The included device-simple example device service demonstrates basic usage of device-sdk-go

device-simple The included device-simple example device service demonstrates basic usage of device-sdk-go. Protocol Driver To make a functional Device

Oct 9, 2021
Server and client implementation of the grpc go libraries to perform unary, client streaming, server streaming and full duplex RPCs from gRPC go introduction

Description This is an implementation of a gRPC client and server that provides route guidance from gRPC Basics: Go tutorial. It demonstrates how to u

Nov 24, 2021
`kawipiko` -- blazingly fast static HTTP server -- focused on low latency and high concurrency, by leveraging Go, `fasthttp` and the CDB embedded database
`kawipiko` -- blazingly fast static HTTP server -- focused on low latency and high concurrency, by leveraging Go, `fasthttp` and the CDB embedded database

kawipiko -- blazingly fast static HTTP server kawipiko is a lightweight static HTTP server written in Go; focused on serving static content as fast an

Jan 3, 2023
An application that uses gRPC Client Streaming framework to computes average.

compute-average An API that uses Client Streaming gRPC framework to capture all integer numbers on Requests that the client sent and returns just the

Dec 26, 2021
IRC bot for launch ddos attack, Mainly of scan target are IoT device that run linux and open default SSH port
IRC bot for launch ddos attack, Mainly of scan target are IoT device that run linux and open default SSH port

IRC bot for launch ddos attack, Mainly of scan target are IoT device that run linux and open default SSH port

Nov 10, 2021
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。

English | ???? 中文 ?? Introduction gnet is an event-driven networking framework that is fast and lightweight. It makes direct epoll and kqueue syscalls

Jan 2, 2023
Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance.
Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance.

Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance. RPC is usually heavy on processing logic and therefore cannot handle I/O serially. But Go's standard library net designed blocking I/O API, so that the RPC framework can only follow the One Conn One Goroutine design.

Jan 2, 2023
High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

Jan 8, 2023
Launch-tui is a small TUI app to manage launchD
Launch-tui is a small TUI app to manage launchD

launch-tui Launch-tui is a small TUI app to manage launchD Install You can either install it by donwloading the archive, uncompress it and add the fol

Sep 27, 2022
Squzy - is a high-performance open-source monitoring, incident and alert system written in Golang with Bazel and love.

Squzy - opensource monitoring, incident and alerting system About Squzy - is a high-performance open-source monitoring and alerting system written in

Dec 12, 2022
🚀Gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.
🚀Gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.

gev 中文 | English gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily bui

Jan 6, 2023
Gmqtt is a flexible, high-performance MQTT broker library that fully implements the MQTT protocol V3.1.1 and V5 in golang

中文文档 Gmqtt News: MQTT V5 is now supported. But due to those new features in v5, there area lots of breaking changes. If you have any migration problem

Jan 5, 2023
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.

aScan A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection. Fu

Aug 15, 2022
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Jan 5, 2023
High performance async-io(proactor) networking for Golang。golangのための高性能非同期io(proactor)ネットワーキング
High performance async-io(proactor) networking for Golang。golangのための高性能非同期io(proactor)ネットワーキング

gaio Introduction 中文介绍 For a typical golang network program, you would first conn := lis.Accept() to get a connection and go func(net.Conn) to start a

Dec 29, 2022
🚥 Yet another pinger: A high-performance ICMP ping implementation build on top of BPF technology.

yap Yet-Another-Pinger: A high-performance ICMP ping implementation build on top of BPF technology. yap uses the gopacket library to receive and handl

Nov 9, 2022