Go Wasm is a in-browser IDE for Go

Go Wasm

Go Wasm is a Go development environment with the essentials to write and run code entirely within the browser, using the power of WebAssembly (Wasm).

Check out the article announcement on Medium, and the demo at https://go-wasm.johnstarich.com

Contributing

Want to discuss an idea or a bug? Open up a new issue and we can talk about it. Pull requests are welcome.

Owner
Comments
  • caching and etags semantics and perhaps wider reuse patterns

    caching and etags semantics and perhaps wider reuse patterns

    Hey @JohnStarich

    Hope all is well.... I am really fascinated with your project and wider implications of this.

    I am not sure if this issue / question shoudl be here or in the FS repo but here goes....

    One thing i am working on is the relationship between retained mode GUI's and data caches. A retained mode gui can be conceptually designed like a data cache system that we all know and love. A gui has a template that is computed on to produce some markup. In a retained mode gui system you want to know if the template has changed. It woudl be a signal at the control plane level (if you get my drift), so you can do optimistic re rendering of that gui component instance, which is sort of like eager caching. Hope that this is making sense :)

    Some background might also help... I know this is pretty out there... The DOM is a browser is doing the retained mode diffing for you. It knows when you changed the HTML in the inspector, and magically updates the HTML Screen for you. But when you working at the WASM level and rendering with WASM to a Webgl canvas, you need to do this yourself..

    So now i get to my question... The FS implementation is super cool because it can work with a WASM, local or remote context, but i was wondering if it supports etags style caching or other meta data semantics to tell the caller that you need to do a cache miss and refetch because your local data is old buddy !!

    Golang Http FS has etags: https://go.googlesource.com/go/+/go1.8.3/src/net/http/fs.go; see line 119. So it can easily act as a cache to a DB called over http for example.

    The reason i bring this up is because you also want to do the same thing inside the WASM environment for many use cases. Let say in the WASM environment you have a FS ( that is indexeddb under the hood) that is sourced from S3. When the S3 source changes, i presume there is some sort of etag header to tell you. So you want when in the WASM environment there is a call to the WASM FS, it should realise that the local FS is now "dirty" or "invalidated" and so it should automatically get the data from S3, update the local file, and return that to the caller.

    So i am wondering if this is currently supported.

    It will be really interesting to see how this also related to the Web Worker and Service Worker stuff too Service Worker.

    It will be really interesting to build quasi CDN like architectures where you can have caches at the WASM, local and remote level like a CDN style architecture. An thew also cross mix. Markup templates are dependent on data which can be dependent on the caache and its "turtles all the way down"..... I like the simplicity of this.

    You also get into a interesting situation with Design time versus Runtime also. I use the work real time because the IDE is running in a browser and doing compilation as you change code. At non real time, you working off already compiled golang. At real time Design time, you want your goimport to be always latest and when someone else PR#s some code that you import you want to know NOW and see it and adapt. you want to get broken by them ... At Real time Runtime, you want to use a static version, and not break.

    The same does for data. YOu have static data ( json, binaries, etc) and you have dynamic data. At Real time Design time, you want everything to be dynamic. Don't use any caching anywhere. At real time Runtime, you want everything that is static to never update, and everything that's dynamic to use the cache mechanism.

  • The building process seems to be very slow

    The building process seems to be very slow

    Hello and thank you for making this! For some reasons I cannot understand, the build process seems to be very slow. I am waiting for more than 5 minutes and still it didn't finish. I wonder if it is browser related;I am using Firefox un GNU/Linux 89 on a 64x machine.

  • Test cases

    Test cases

    This definitely is a fascinating project. The docker ceo apparently said that if WASM existed there would have been no need for docker. He was referring to sand boxing.

    Did you hit many bugs when attempting compilation using tinygo. I think they did a release recently or it might still be on a branch . Tinygo is efficient enough that you could run on mobile or have a good chance.

    I have a interesting idea. Your sage accounting software uses a bucket file system dB and a git based vcs to presumerable allow sync of the underlying files . Would be interesting test case if these two work running on top of the wasm indexdb / FS. It’s a decent test case I feel.

    Would badger dB in theory work ?

  • Race condition in pub sub

    Race condition in pub sub

    https://github.com/JohnStarich/go-wasm/blob/fd0edf41a665e400fef3db6616772439ce4b9738/internal/pubsub/pubsub.go#L40

    Go routines A and B call emit(). Both see that visited == false under read lock. A acquires write lock and processes. B acquired right lock and double processes, even though A marked the node as visited. The fix is to check if visited again after write lock is acquired

  • Seeing error

    Seeing error "package fmt is not in GOROOT (/usr/local/go/src/fmt)" in attempting hello world on hackpad.org

    Seeing the following error, see linked screenshot below: Screenshot 2022-05-06 4 08 51 PM

    Terminal text here for convenience also:

    $ go version
    go version go1.16.6 js/wasm
    $ go mod init playground
    go: creating new go.mod: module playground
    go: to add module requirements and sums:
    	go mod tidy
    $ go mod tidy
    $ go build -v .
    main.go:4:2: package fmt is not in GOROOT (/usr/local/go/src/fmt)
    package playground: cannot find package
    exit status 1
    

    Any ideas what I might be doing wrong here? Thanks!

  • Switch filesystem to hackpadfs

    Switch filesystem to hackpadfs

    Switches to hackpadfs for filesystem setup, tests, and implementations. Parts of its original ideas were extracted from go-wasm, but it evolved into a totally new implementation.

    Hackpadfs has proper tests, interface design, and documentation. All of which should help improve stability, isolation, and performance of go-wasm in the long run.

    Further, the indexeddb adapter is much improved in its new home go-indexeddb.

  • Add vet and cover tools

    Add vet and cover tools

    Add vet and cover go tool support.

    Also add a jsdownload builtin command for the shell. This makes it possible to generate coverage files for Wasm-only tests and then parse them on your own machine.

    i.e.

    # in Go Wasm
    go test ./... -coverprofile=cover.out
    jsdownload cover.out
    # locally
    go tool cover -html cover.out
    

    Perhaps eventually coverage can be fully integrated into the editor itself.

  • Fix overlapped vertical scrollbar

    Fix overlapped vertical scrollbar

    When one go beyond the editor viewport, the scrollbar overlaps the original one (which doesn't seem to have any utility). So, in order to avoid this situation, the 'original' vertical scrollbar has been remove.

    Before: before

    After: after

  • Lower memory usage from compiling and running binaries

    Lower memory usage from compiling and running binaries

    Lowers memory usage for running binaries like go, compile, and user programs.

    Memory decreased by up to 56% in the best case and 47% in the worst case. The heap footprint should now remain below 250 MB in the worst case, down from 540 MB. On a second boot (not installing the Go toolchain), memory usage is reduced even further to below 180 MB. Execution times shortened a bit, but probably within margin of error.

    See below for single samples of performance changes. These are super rough, so take them with a grain of salt.

    Master branch:

    • First boot:
      • Init = 59.04s; 297 MB heap
      • Compile = 179.18s
      • Run = 0.20s; 540 MB heap
    • Second boot (after reload):
      • Init = 0.85s; 224 MB heap
      • Compile = 27.95s
      • Run = 0.12s; 394 MB heap

    This PR:

    • First boot:
      • Init = 64.35s; 156 MB heap -47%
      • Compile = 164.41s
      • Run = 0.13s; 245 MB heap -55%
    • Second boot:
      • Init = 0.94s; 104 MB heap
      • Compile = 25.06s
      • Run = 0.10s; 175 MB heap -56%

    I'd summarize the commits, but they were enough changes it'd probably be best to read through the commit messages themselves.

  • Shell auto-completion, editor DOM interface pkg & settings dropdown

    Shell auto-completion, editor DOM interface pkg & settings dropdown

    • Upgrade to Go 1.16 (compiler & runtime compiler)
    • Improve mount handling for long-running FS calls
    • Add new dom package for clearer interop and compile safety in the editor
    • Add basic auto-completion for file paths in the shell
    • Add support for ~ meaning /home/me in the shell
    • Add settings dropdown in editor for easy IDB FS management, like dropping the build cache or Go installation
  • Add and use IndexedDB file system to reduce memory usage

    Add and use IndexedDB file system to reduce memory usage

    The net of all of the below changes is: Significantly reduced memory usage and persistent files. Typical memory usage has dropped from ~750 MB to ~250 MB. This is a big leap architecturally, but we still have room to improve going forward. πŸŽ‰

    Since the outrageous memory usage in the current release causes crashes on typical devices, we needed to store files somewhere other than RAM. In switching to a non-RAM storage medium, naturally go-wasm slowed down a bit. This speed difference can be further reduced in future PRs.

    As a bonus, Safari now almost works as well as Chrome and Firefox. It's still rather unhappy with the memory usage, though. Pro tip: Safari basically hangs if you run WebAssembly.compile around 100 times on the same data. Best to avoid that.

    • Fixes critical bugs in the previously experimental IndexedDB file system.
    • Overlays IDB FS's on the Go installation directory, the home dir (editor files), and the system cache dir.
      • Reconfigures Go Modules cache directory to sit on the system cache dir, too.
    • Heavily optimizes both IDB FS and tarfs for execution speed. In particular, utilizes "batched parallelism" to queue up large numbers of file operations and completing them all in one pass on the FS end.
    • Rips out tarfs custom file storage and replaced with an afero.Fs interface. Enabled easy swap-in of IDB FS.
    • New blob.Blob abstraction, to significantly reduce conversion costs between []byte and Uint8Array
    • Adds a bit of a hack to support synchronous chdir on a strictly asynchronous valid directory file system check. It works, but it ain't great.
    • Adds shell history in ~/.history. Persisted with home dir IDB FS.
    • Fixes compatibility with Go 1.16beta1, namely adding go mod tidy to first boot.
    • Fixes PATH environment variable propagation.
    • Fleshes out Go memory profile downloads a bit.
    • Adds JS property name caching, since decode() is surprisingly slow, relative to everything else.
    • Fixes an issue with stdout/stderr new lines not rendering correctly in the terminal.
    • Makes sh more robust, with time and touch support, plus a new long format -l flag for ls .
    • Fixes file timestamp conversions. (It helps when "today" is even remotely in this decade.)

    Thanks to @guest271314 for the helpful nudge #4 ❀️

  • Compiler as a Service

    Compiler as a Service

    It would be cool to have a compiler as a service running on a server that the HackPad IDE can interact with.

    the reason is because I am interested in making plugins . You write it in golang and compile with Tinygo . The Compiler As A Service ( CAAS ) does that and then you can β€œ see it β€œ in the IDE.

    I saw the Web Worker Par in HackPadFS and it got me thinking. I presume that’s so that the FS in running isolated in its own isolated area outside of the main GUI and so hence hardening the system from crashes.

    so my rational is to do it for all code. The developers code that they are developing in the IDE is compiled and exposed as a Web Worker.

    The same can then be done with Wazero so that you can run the wasm on the server also. Why think about this ? Because you end up with reconfigurable ( is that a real word ? ) computing, and so depending on the nature of the code can run it in a Browser or on a Server ( with Wazero ).

    The way WASI and WASM exposes the exports is different . But it’s possible to shim that by writing a compile time wrapping that walks the AST and code generates the equivalents needs for it to run in the browser too. If we write the exports out as a JSON file then an IDE can β€œ discover β€œ the exports and elevate it in the IDE GUI.

    @codefromthecrypt as I figure this is interesting to you .

    I probably should have made this a discussion.

    My request is to know what are the blockers. is my codegen concept the way to go for at least the golang community. I read about WIX but the more I delve into it I get the impression that there is no standard yet ( or ever will be ) for WASI and WASM being able to be single sourced and run in both environment easily . Hence sone compile time and runtime wrapper ingredients seems needed.

    The isolation aspects of WebWorker and Service Workers can possibly mapped to similar topology primitives on the Server works. A WebWorker is easy because it’s the same as a Fly.io Machine or a Wazero server hosting WASI. the Service Worker equivalent would be a gateway / Proxy / Caching type of thing similar to Caddy with Caddy Modules but not sure . Service workers have logic , caching and procuring capabilities.

  • [RFC] refactor: update to go 1.19

    [RFC] refactor: update to go 1.19

    Marked RFC until these are merged:

    • https://github.com/paralin/go/tree/hackpad/release-branch.go1.19
    • https://github.com/hack-pad/hackpadfs/pull/20

    Fixes #25

  • Editing the editor (tutorial platform)

    Editing the editor (tutorial platform)

    Hey, This is super cool. I have wanted for quite some time to build tutorials for the Go language using wasm (in browser) - you have probably jump started my work here by a mile. The first step is to make a super interactive platform and a 'framework' for the tutorials so its fast and easy to add new ones.

    Thoughts:

    • left hand side panel where the instructions for this step/tutorial are
    • pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.
    • tutorials can use libraries that already exist (your platform offers this)
    • I can use mockers (smocker) to mock out external services in tutorials
    • I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

    Questions

    left hand side panel where the instructions for this step/tutorial are

    I looked through the code and see you are designing/rendering the UI in Go which was amazing/surprising. Two questions really - one is do you have a link to tutorial in how you are doing this, and secondly, could this be swapped out for HTML/CSS/JS or React or something easily? My guess is WASM means better to do all in Go, so question one is more the focus - i.e can you point me to anything to learn how to adjust the UI?

    pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.

    This sounds possible based on your "OS" - Would be cool if tests can run against code?

    I can use mockers (smocker) to mock out external services in tutorials I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

    I suspect these last two are closely related and its either "thats easy" or "thats hard" as the OS isn't really "real".....

    Anyway would love just to hear your thoughts on this usecase for this platform. Thanks

  • cant run

    cant run "web-workers-2" branch

    Had some time to jump into this rabbit hole again and playing around with File System explorer module, so you get a tree.

    On origin/feature/web-workers-2 branch

    This is my simple make script that wraps your make script..

    "make serve" does not work for me. Any ideas ?

    
    build-init:
    	# installs special go.
    	cd $(REPO_NAME) && $(MAKE) go
    
    build:
    	# builds the many wasm and then the server npm stuff.
    	cd $(REPO_NAME) && $(MAKE) build
    
    serve:	
    	cd $(REPO_NAME) && $(MAKE) serve
    
  • Run processes in web workers, use message passing for orchestration

    Run processes in web workers, use message passing for orchestration

    Closes https://github.com/hack-pad/hackpad/issues/11

    This is currently a huge refactor and a work in progress. Hopefully will suss out the major bugs soon πŸ˜…

Run WASM tests inside your browser

wasmbrowsertest Run Go wasm tests easily in your browser. If you have a codebase targeting the wasm platform, chances are you would want to test your

Dec 16, 2022
The in-browser IDE for Go

Go Wasm Go Wasm is a Go development environment with the essentials to write and run code entirely within the browser, using the power of WebAssembly

Dec 19, 2022
DOM library for Go and WASM

Go DOM binding (and more) for WebAssembly This library provides a Go API for different Web APIs for WebAssembly target. It's in an active development,

Dec 23, 2022
Library to use HTML5 Canvas from Go-WASM, with all drawing within go code

go-canvas go-canvas is a pure go+webassembly Library for efficiently drawing on a html5 canvas element within the browser from go without requiring ca

Dec 11, 2022
An Experimental Wasm Virtual Machine for Gophers

gasm A minimal implementation of v1 WASM spec compatible virtual machine purely written in go. The vm can be embedded in your go program without any d

Dec 31, 2022
Golang-WASM provides a simple idiomatic, and comprehensive API and bindings for working with WebAssembly for Go and JavaScript developers
Golang-WASM provides a simple idiomatic, and comprehensive API and bindings for working with WebAssembly for Go and JavaScript developers

A bridge and bindings for JS DOM API with Go WebAssembly. Written by Team Ortix - Hamza Ali and Chan Wen Xu. GOOS=js GOARCH=wasm go get -u github.com/

Dec 22, 2022
A WASM Filter for Envoy Proxy written in Golang

envoy-proxy-wasm-filter-golang A WASM Filter for Envoy Proxy written in Golang Build tinygo build -o optimized.wasm -scheduler=none -target=wasi ./mai

Nov 6, 2022
Istio wasm api demo with golang

istio-wasm-api-demo 1. Setup the latest Istio Setup k8s cluster: e.g. kind create cluster --name test Download the latest Istioctl from the GitHub rel

Nov 1, 2022
Go compiler running entirely in your browser

wasm-go-playground This is the Go compiler ("gc") compiled for WASM, running in your browser! It can be used to run a simple playground, Γ  la play.gol

Nov 10, 2022
Interact with browser from Go. Manually-crafted WebAPI interoperation library.

GWeb: golang + js + wasm gweb -- strictly typed WebAPI library on top of syscall/js. Like flow or TypeScript but for Go. You need it if you want to in

Sep 26, 2022
Running a Command line tool written in Go on browser with WebAssembly

Running a command line tool written in Go on browser with WebAssembly This repo contains code/assets from the article Files: . β”œβ”€β”€ article.md

Dec 30, 2022
Aes for go and java; build go fo wasm and use wasm parse java response.

aes_go_wasm_java aes for go and java; build go fo wasm and use wasm parse java response. vscode setting config settings.json { "go.toolsEnvVars":

Dec 14, 2021
Run WASM tests inside your browser

wasmbrowsertest Run Go wasm tests easily in your browser. If you have a codebase targeting the wasm platform, chances are you would want to test your

Dec 16, 2022
The in-browser IDE for Go

Go Wasm Go Wasm is a Go development environment with the essentials to write and run code entirely within the browser, using the power of WebAssembly

Dec 19, 2022
hack-browser-data is an open-source tool that could help you decrypt data from the browser.
hack-browser-data is an open-source tool that could help you decrypt data  from the browser.

hack-browser-data is an open-source tool that could help you decrypt data ( password|bookmark|cookie|history|credit card|download

Dec 23, 2022
🌭 The hotdog web browser and browser engine 🌭
🌭 The hotdog web browser and browser engine 🌭

This is the hotdog web browser project. It's a web browser with its own layout and rendering engine, parsers, and UI toolkit! It's made from scratch e

Dec 30, 2022
DOM library for Go and WASM

Go DOM binding (and more) for WebAssembly This library provides a Go API for different Web APIs for WebAssembly target. It's in an active development,

Dec 23, 2022
Library to use HTML5 Canvas from Go-WASM, with all drawing within go code

go-canvas go-canvas is a pure go+webassembly Library for efficiently drawing on a html5 canvas element within the browser from go without requiring ca

Dec 11, 2022
An Experimental Wasm Virtual Machine for Gophers

gasm A minimal implementation of v1 WASM spec compatible virtual machine purely written in go. The vm can be embedded in your go program without any d

Dec 31, 2022
A small fantasy game engine in WASM using GoLang
A small fantasy game engine in WASM using GoLang

The GoLang Fantasy Engine (GoLF Engine) is a retro game engine. It draws inspiration from fantasy console projects like pico-8, tic-80, and pyxle. Like those projects it is designed to be a retro-feeling game creation/playing tool. Unlike those projects GoLF is more minimal in scope and only provides an API and a small set of tools to help you create your games. Tools like an image editor and code editor are not built in. Despite this minimalism creating games in GoLF is still easy and should still maintain the retro game feel.

Jul 16, 2022