🕸️ Gossamer: A Go implementation of the Polkadot Host

Gossamer logo

A Go Implementation of the Polkadot Host

Gossamer is an implementation of the Polkadot Host: a framework used to build and run nodes for different blockchain protocols that are compatible with the Polkadot ecosystem. The core of the Polkadot Host is the wasm runtime which handles the logic of the chain.

Gossamer includes node implementations for major blockchains within the Polkadot ecosystem and simplifies building node implementations for other blockchains. Runtimes built with Substrate can plug their runtime into Gossamer to create a node implementation in Go.

For more information about Gossamer, the Polkadot ecosystem, and how to use Gossamer to build and run nodes for various blockchain protocols within the Polkadot ecosystem, check out the Gossamer Docs.

Get Started

Prerequisites

install go version >=1.17

Installation

get the ChainSafe/gossamer repository:

git clone [email protected]:ChainSafe/gossamer
cd gossamer

build gossamer command:

make gossamer

Troubleshooting for Apple Silicon users

If you are facing the following problem with the wasmer:

undefined: cWasmerImportObjectT
undefined: cWasmerImportFuncT
undefined: cWasmerValueTag

Make sure you have the following Golang enviroment variables:

  • GOARCH="amd64"
  • CGO_ENABLED="1"

use go env to see all the Golang enviroment variables

use go env -w ENV_NAME=ENV_VALUE to set the new value

Run Development Node

To initialise a development node:

./bin/gossamer --chain dev init

To start the development node:

./bin/gossamer --chain dev

The development node is configured to produce a block every slot and to finalise a block every round (as there is only one authority, alice.)

Run Gossamer Node

The gossamer node runs by default as an authority with 9 authorites set at genesis. The built-in keys, corresponding to the authorities, that are available for the node are alice, bob, charlie, dave, eve, ferdie, george, and ian.

To initialise a gossamer node:

./bin/gossamer --chain gssmr init

To start the gossamer node:

./bin/gossamer --chain gssmr --key alice

Note: If you only run one gossamer node, the node will not build blocks every slot or finalize blocks; it will appear that the node is doing nothing, but it is actually waiting for a slot to build a block. This is because there are 9 authorities set, so at least 6 of the authorities should be run for a functional network. If you wish to reduce the number of authorities, you can modify the genesis file in chain/gssmr/genesis-spec.json.

Run Kusama Node

Kusama is currently supported as a full node, ie. it can sync the chain but not act as an authority.

To initialise a kusama node:

./bin/gossamer --chain kusama init

To start the kusama node:

./bin/gossamer --chain kusama

The node may not appear to do anything for the first minute or so (it's bootstrapping to the network.) If you wish to see what is it doing in this time, you can turn on debug logs in chain/gssmr/config.toml:

[log]
network = "debug"

After it's finished bootstrapping, the node should begin to sync.

Run Polkadot Node

Polkadot is currently supported as a full node, ie. it can sync the chain but not act as an authority.

To initialise a polkadot node:

./bin/gossamer --chain polkadot init

To start the polkadot node:

./bin/gossamer --chain polkadot

Contribute

Donate

Our work on gossamer is funded by grants. If you'd like to donate, you can send us ETH or DAI at the following address: 0x764001D60E69f0C3D0b41B0588866cFaE796972c

ChainSafe Security Policy

Reporting a Security Bug

We take all security issues seriously, if you believe you have found a security issue within a ChainSafe project please notify us immediately. If an issue is confirmed, we will take all necessary precautions to ensure a statement and patch release is made in a timely manner.

Please email us a description of the flaw and any related information (e.g. reproduction steps, version) to security at chainsafe dot io.

License

GNU Lesser General Public License v3.0


Owner
Comments
  • Trie root hash does not match the result from rust implementation

    Trie root hash does not match the result from rust implementation

    $ ./run_tests.sh
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /home/user/polkadot-re-tests/build
    [100%] running cargo
        Finished dev [unoptimized + debuginfo] target(s) in 0.77s
    [100%] Built target rust_tester_target
    [100%] Built target go_tester
    [100%] Built target go_tester_all
    [100%] running cargo
        Finished dev [unoptimized + debuginfo] target(s) in 0.13s
    [100%] Built target rust_tester_target
    [100%] Built target go_tester
    [100%] Built target go_tester_all
    Install the project...
    -- Install configuration: ""
    -- Up-to-date: bin/usr/local/bin/rust_tester
    -- Installing: bin/usr/local/bin/go_tester
    Test Summary:     | Pass  Total
    Scale Codec Tests |    6      6
      2.552807 seconds (4.40 M allocations: 226.640 MiB, 4.80% gc time)
    WARNING: replacing module CommonFixtures.
    WARNING: replacing module ScaleCodecFixtures.
    WARNING: replacing module StateTrieFixtures.
    state trie root: 0xa10270d88debdb7005451775b908c2d575a37bfed94daaa0b6fa14045372e45c
    
    state root hash: 0x4213043100000000000000000000000000000000000000000000000000000000
    State Trie Tests: Test Failed at /home/user/polkadot-re-tests/test/state_trie_tests.jl:18
      Expression: test_result_array[end] == test_result_array[CommonFixtures.reference_implementation]
       Evaluated: "state root hash: 0x4213043100000000000000000000000000000000000000000000000000000000" == "state trie root: 0xa10270d88debdb7005451775b908c2d575a37bfed94daaa0b6fa14045372e45c\n"
    Stacktrace:
     [1] top-level scope at /home/user/polkadot-re-tests/test/state_trie_tests.jl:18
     [2] top-level scope at /build/julia/src/julia-1.1.0/usr/share/julia/stdlib/v1.1/Test/src/Test.jl:1083
     [3] top-level scope at /home/user/polkadot-re-tests/test/state_trie_tests.jl:5
    state trie root: 0x7cdd17212f13b5943c04cc48c59a4e2307f4b4f6370a4f10db2195b5ae90589c
    
    state root hash: 0xb804236613fdec5bec79e990892cb84c7941af6cdd9879917221ca0655970f15
    State Trie Tests: Test Failed at /home/user/polkadot-re-tests/test/state_trie_tests.jl:18
      Expression: test_result_array[end] == test_result_array[CommonFixtures.reference_implementation]
       Evaluated: "state root hash: 0xb804236613fdec5bec79e990892cb84c7941af6cdd9879917221ca0655970f15" == "state trie root: 0x7cdd17212f13b5943c04cc48c59a4e2307f4b4f6370a4f10db2195b5ae90589c\n"
    Stacktrace:
     [1] top-level scope at /home/user/polkadot-re-tests/test/state_trie_tests.jl:18
     [2] top-level scope at /build/julia/src/julia-1.1.0/usr/share/julia/stdlib/v1.1/Test/src/Test.jl:1083
     [3] top-level scope at /home/user/polkadot-re-tests/test/state_trie_tests.jl:5
    Test Summary:      | Fail  Total
    State Trie Tests   |    2      2
      State Trie Tests |    2      2
    ERROR: LoadError: Some tests did not pass: 0 passed, 2 failed, 0 errored, 0 broken.
    in expression starting at /home/user/polkadot-re-tests/test/runtests.jl:3
    
    
  • fix(trie): decode inline child nodes

    fix(trie): decode inline child nodes

    Changes

    • Adds handler for decoding inline child nodes based on expected hash length.
    • Achieves behavioral parity with the parity implementation here: https://github.com/paritytech/substrate/blob/542d587c3db7e46b6fff423e8efc4f327f7b79a1/primitives/trie/src/node_codec.rs#L141-L145

    Tests

    • Adds a test for decoding a branch with inlined child nodes.

    Issues

    • Fixes proof issue: https://github.com/ChainSafe/gossamer/issues/2329

    Primary Reviewer

  • cmd, dot: migrate node logic to dot

    cmd, dot: migrate node logic to dot

    Changes

    • migrate configuration logic from command package to dot package
    • migrate keystore logic from command package to keystore package
    • create test utils package and implement in cmd and dot root level

    Tests:

    go clean --testcache ./cmd/...
    go test ./cmd/...
    
    go clean --testcache ./dot
    go test ./dot
    
    go clean --testcache ./lib/keystore
    go test ./lib/keystore
    
    go clean --testcache ./lib/utils
    go test ./lib/utils
    
    go clean --testcache ./...
    go test ./...
    

    build and run gossamer command:

    make gossamer
    ./bin/gossamer init --key alice --datadir tmp/alice --verbosity trace
    ./bin/gossamer --key alice --datadir tmp/alice --verbosity trace
    

    Issues:

    closes #252

  • chore (pkg/scale) types refactor to utilize new scale pkg

    chore (pkg/scale) types refactor to utilize new scale pkg

    Changes

    • This is a refactor of the types used in the dot package. Includes changes to digestItem, digest, header, block data, block, and grandpa (grandpa is minor currently).
    • This PR also includes a scale integration into the Digest and Network packages
    • Note: Some of the files have a mix of old and new scale. I didn't want to get carried away with what I did in this PR, so I plan to continue with unfinished scale integrations in future PRs, unless it's suggested I do otherwise.

    Tests

    go test ./dot/types -v
    go test ./dot/network -v -short
    go test ./dot/digest -v -short
    go test ./pkg/scale -v -short
    

    Issues

    • #1621
  • testing(dot): create dot unit tests

    testing(dot): create dot unit tests

    Changes

    • Create unit tests dot package: build_spec_test.go, config_test.go, import_test.go, node_test.go, service_test.go and utils_test.go

    Tests

    go test ./dot/ -v
    

    Issues

    • closes #1952

    Primary Reviewer

    • @timwu20
  • Implement external flags & Local defaults

    Implement external flags & Local defaults

    Changes

    • --rpc flag changed to --rpc-external
    • --ws flag changed to --ws-external

    Tests

    
    

    Checklist

    • [x] I have read CODE_OF_CONDUCT and CONTRIBUTING
    • [x] I have provided as much information as possible and necessary
    • [x] I have reviewed my own pull request before requesting a review
    • [ ] All integration tests and required coverage checks are passing

    Issues

    closes #864

  • cmd, config, runtime, trie: load genesis state into storage trie and load `:code`

    cmd, config, runtime, trie: load genesis state into storage trie and load `:code`

    Changes

    • added functionality to load the state from the genesis file and load it into the trie, trie is also written to new StateDB
    • created a GenesisState type which holds the loaded state (and later p2p config)
    • makeNode takes the GenesisState and loads the runtime code from the trie at :code
    • in runtime, added NewRuntimeFromFile (same as NewRuntime previously) and changed NewRuntime to take code bytes instead

    Tests:

    go test ./... -short
    

    Issues:

    closes #339 closes #322

  • fix(ci): embed v0.9.20 runtime, update test suite, and ci workflows

    fix(ci): embed v0.9.20 runtime, update test suite, and ci workflows

    Changes

    • embeds 0.9.20 runtime
    • updated unit tests that require old runtimes (gssmr-v3substate, dev-v3substrate)
    • updated all ci workflows
    • updated integration tests

    Tests

    go test -tags integration github.com/ChainSafe/gossamer
    

    Issues

    completes #2540

    Primary Reviewer

    @jimjbrettj

  • chore(logging): replace `log15` library

    chore(logging): replace `log15` library

    Changes

    💁 First and foremost, my apologies for the rather huge PR. Although it's mostly trivial changes, clearing up some logs.

    • [x] Add an internal/log package (initially tried to use zerolog, then standard log, then just re-invented the wheel since we needed custom caller code for our runtime packages - although that was rather quick to develop)
    • [x] Has features we used from the log15 logger except context key values
    • [x] Can still be used with a global scope and thread safely (although we should inject it ideally)
    • [x] Thread safety logging issues fixed. They now use all the same mutex 100% sure.
    • [x] Adapt all code to not use the older context key-values
      • [x] Review most log instructions and check if they make sense (doing it at the same time)
    • [x] Some thread safety issues fixed (was using different logs all writing to stdout)
    • [x] internal/log tests
      • [x] caller_test.go
      • [x] level_test.go
      • [x] log_test.go
      • [x] logger_test.go
      • [x] settings, options are well tested (full coverage) by the tests above
      • [x] patch_test.go
      • [x] global_test.go
      • [x] Data races tests (to protect it if changing it further in the future)
    • [x] ~Modify runtime/wasmer when my language server is willing to compile it. RESOLVED, WIP~
    • [x] Change all logging functions using fmt.Sprintf to formatters (e.g. Debug to Debugf using fmt.Sprintf to delay formatting depending on level)
    • [x] ~Removed coloring for now since some shows as CRIT[10-28|15:12:03] [... in some terminals which is extremely confusing~ Change colouring to be 'clever' with fatih/color.
    • [x] Move log implementation in a subpackage so we can have more implementations (e.g. test oriented logger)
    • [ ] Question ⁉️ Do we actually care about the logs in tests? Can we change all use of the loggers by tests to have the writer io.Discard. One can always change it to os.Stdout if some debugging is needed.
    • [x] Self review 126/126
    • [x] Create issues
      • [x] Inject the logger everywhere instead of using the global one: #1946
      • [x] ~Disable logger coloring with flags etc.~
      • [x] String methods needed: #1945

    Tests

    CGO_ENABLED=1 go test -race github.com/ChainSafe/gossamer/internal/log/...
    

    Issues

    • #1845

    Primary Reviewer

    • None for now
  • chore(dot/core, dot/sync, lib/babe): refactor handling on block import to reduce duplicate code

    chore(dot/core, dot/sync, lib/babe): refactor handling on block import to reduce duplicate code

    Changes

    • when a block is imported into the chain, either via from syncing a block from the network or importing a block we produced, various updates and checks need to happen, namely:
    1. storing block in db
    2. storing state trie in db
    3. checking for runtime changes
    4. checking for code substitutions
    5. handling consensus digests
    • previously this was happening in two different places (sync and babe) and in the case of babe some checks weren't happening at all
    • this PR refactors the handling of an imported block to go through core.Service via the HandleBlockImport or HandleBlockProduced funtions for sync and babe respectively
    • also, moved core.DigestHandler into its own package dot/digest: core.DigestHandler -> digest.Handler
    • cleanup relics of interfaces no longer needed by core, sync, and babe

    Tests

    go test ./dot/core
    go test ./dot/sync -short
    go test ./lib/babe
    

    Issues

    • related to #1595, related to #1624
  • refactor: docs: Docs Overhaul

    refactor: docs: Docs Overhaul

    Review instructions

    The contents of each page can be edited at the HackMD link attached to each, for where said files reside in this branch, the path is provided

    Changes

    Getting Started

    • [x] Overview - docs/docs/index.md
    • [x] Installation - docs/docs/getting-started/installation.md
    • [x] Host Architecture - docs/docs/getting-started/overview/host-architecture.md
    • [x] Package lib - docs/docs/getting-started/overview/package-library.md

    Resources

    Usage

    Integrate

    Testing & Debugging @dutterbutter

    Deployment

    Advanced

    • [x] SCALE - docs/docs/advanced/scale-examples.md

    Contributing

    • [x] Overview - docs/docs/contibuting.md

    Tests

    
    

    Checklist

    • [ ] I have read CODE_OF_CONDUCT and CONTRIBUTING
    • [ ] I have provided as much information as possible and necessary
    • [ ] I have reviewed my own pull request before requesting a review
    • [ ] All integration tests and required coverage checks are passing

    Issues

    • closes #1357
    • closes #1214
    • closes #1213
    • closes #1357
    • closes #918
    • closes #868
    • closes #867
    • closes #710
  • tests failing with `Timestamp slot must match 'CurrentSlot'` while using westend-dev spec file

    tests failing with `Timestamp slot must match 'CurrentSlot'` while using westend-dev spec file

    The following tests are failing while using the westend-dev spec file:

    • TestChainProcessor_HandleBlockResponse_ValidChain
    • TestChainProcessor_HandleBlockResponse_MissingBlocks
    • TestChainProcessor_HandleBlockResponse_BlockData
    • TestChainProcessor_ExecuteBlock

    Those tests should be able to use the westend-dev spec file

  • `BlockBuilder_apply_extrinsic` fails in `TestNodeRuntime_ValidateTransaction` while using westend-dev spec file

    `BlockBuilder_apply_extrinsic` fails in `TestNodeRuntime_ValidateTransaction` while using westend-dev spec file

    In order to address the epic https://github.com/ChainSafe/gossamer/issues/2895 we should replace the usage of GetGssmrV3SubstrateGenesisRawPathTest to GetWestendDevGenesisPath but in the test function TestNodeRuntime_ValidateTransaction when we use the GetWestendDevGenesisPath we got the following error:

    target=runtime message=panicked at 'Bad input data provided to apply_extrinsic: Codec error', /builds/runtime/westend/src/lib.rs:1276:1
    

    and the test fails with the following assertion problem:

    exports_test.go:350:
            	Error Trace:	/ChainSafe/gossamer/lib/runtime/wasmer/test_helpers.go:340
            	      /ChainSafe/gossamer/lib/runtime/wasmer/exports_test.go:350
            	Error:      	Received unexpected error:
            	            	running runtime function: Failed to call the `BlockBuilder_apply_extrinsic` exported function.
            	Test:       	TestNodeRuntime_ValidateTransaction
            	Messages:   	[20 4 2 0 225 46]
    

    We should be able to run this test against the westend-dev spec file without errors

  • chore(utils): replace all the unneeded spec usage for `westend-dev` runtime

    chore(utils): replace all the unneeded spec usage for `westend-dev` runtime

    Changes

    • Replace the usage of all GetGssmrGenesis* utils functions for GetWestendDevGenesisPath() in order to remove unneeded spec usage

    Tests

    N/A

    Issues

    • Ref: https://github.com/ChainSafe/gossamer/issues/2895

    Primary Reviewer

  • remove the call to `GetGssmrGenesisPath` from `createGenesisWithRuntime`

    remove the call to `GetGssmrGenesisPath` from `createGenesisWithRuntime`

    Currently, the function createGenesisWithRuntime (cmd/gossame/import_runtime) calls the utils.GetGssmrGenesisPath().

    In order to address the epic https://github.com/ChainSafe/gossamer/issues/2895 we should update this function to don't call the GetGssmrGenesisPath function

  • chore(lib/wasmer): `NewTestInstanceWithTrie` checks if `targetRuntime` is a valid filepath

    chore(lib/wasmer): `NewTestInstanceWithTrie` checks if `targetRuntime` is a valid filepath

    Changes

    • NewTestInstanceWithTrie is a function that given a target runtime it builds a wasmer instance for testing, now this function will check if the target runtime is a valid file path, if so we will use it to load the wasmer instance otherwise we will try to fetch it remotely.

    Tests

    N/A

    Issues

    • Closes https://github.com/ChainSafe/gossamer/issues/3010

    Primary Reviewer

    @timwu20

WebAssembly for Proxies (Golang host implementation)

WebAssembly for Proxies (GoLang host implementation) The GoLang implementation for proxy-wasm, enabling developer to run proxy-wasm extensions in Go.

Dec 29, 2022
Cross-platform Go library to place an icon in the host operating system's taskbar.

trayhost Package trayhost is a cross-platform Go library to place an icon in the host operating system's taskbar. Platform Support macOS - Fully imple

Nov 6, 2022
Host yo' self from your browser, your phone, your toaster.
Host yo' self from your browser, your phone, your toaster.

A hosting service from the browser, because why not. Try it at hostyoself.com. See it in action Here's an example where I use hostyoself.com to host i

Jan 1, 2023
Go library for accessing multi-host SQL database installations

hasql hasql provides simple and reliable way to access high-availability database setups with multiple hosts. Status hasql is production-ready and is

Dec 28, 2022
Use SQL to query host, DNS and exploit information using Shodan. Open source CLI. No DB required.

Shodan Plugin for Steampipe Query Shodan with SQL Use SQL to query host, DNS and exploit information using Shodan. For example: select * from shod

Nov 10, 2022
High-Performance Shortlink ( Short URL ) app creator in Golang. For privacy reasons, you may prefer to host your own short URL app and this is the one to use.
High-Performance Shortlink ( Short URL ) app creator in Golang. For privacy reasons, you may prefer to host your own short URL app and this is the one to use.

About The Project Shortlink App in Golang Multiple Node based Architecture to create and scale at ease Highly performant key-value storage system Cent

Jan 3, 2023
EasyAgent is an infrastructure component, applied to manage the life-cycle of services on the remote host.
EasyAgent is an infrastructure component, applied to manage the life-cycle of services on the remote host.

Easyagent English | 中文 介绍 easyagent是在袋鼠云内部广泛使用的基础架构组件,最佳应用场景包括ELK体系beats等数据采集器的管控和配置管理、数栈体系自动化部署等 基本原理 easyagent主要有sidecar和server两个组件,sidecar部署在主机端,si

Nov 24, 2022
LazySSH is an SSH server that acts as a jump host only, and dynamically starts temporary virtual machines.

LazySSH is an SSH server that acts as a jump host only, and dynamically starts temporary virtual machines. If you find yourself briefly starti

Dec 11, 2022
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Dec 30, 2022
Rhoas CLI as an Host example
Rhoas CLI as an Host example

RHOAS CLI rhoas is a command-line client for managing all of your application services on Kubernetes. Guides See our Guides for installation and usage

Nov 8, 2021
Receive phone calls from anybody on the Web. Or host a telephony server yourself.
Receive phone calls from anybody on the Web. Or host a telephony server yourself.

WebCall WebRTC Telephony Server Browser based telephony over E2E-encrypted P2P-links with very high audio quality. WebCall is lightweight and easy to

Jan 5, 2023
⚡ 🖥️ 👾 Host your own Lightning Address on LND

⚡ ??️ ?? Host your own Lightning Address on LND Lighting Wallets like BlueWallet, Blixt and many more allow us to send sats to Lighting Addresses like

Dec 22, 2022
Make any web accessible from your own host / domain
Make any web accessible from your own host / domain

Web Mirror Based on reverseproxy Solution for: Website that only set 'X-Frame-Options' to 'sameorigin'. Hide website real url Content interception & m

May 31, 2022
Proxy your Go Module`s Import Path from your own domain to a public host (e.g. github.com).

Go Modules Remote Import Path Proxy Proxy your Go Module`s Import Path from your own domain to a public host (e.g. github.com). For example Uber (buil

Nov 2, 2021
htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.
htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.

htf htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster. All you need to do is populate the htf configura

Nov 28, 2021
The rest api that can manage the iptables rules of the remote host

fiewall-api firewall api是基于firewalld来远程管理iptables规则的rest-api,无需部署agent Features 指定一个主机ip,让这个主机上的iptables增加一个规则 处理单个IP或CIDR范围(xx.xx.xx.xx/mask,mac,inte

Mar 24, 2022
ARP spoofing tool based on go language, supports LAN host scanning, ARP poisoning, man-in-the-middle attack, sensitive information sniffing, HTTP packet sniffing
ARP spoofing tool based on go language, supports LAN host scanning, ARP poisoning, man-in-the-middle attack, sensitive information sniffing, HTTP packet sniffing

[ARP Spoofing] [Usage] Commands: clear clear the screen cut 通过ARP欺骗切断局域网内某台主机的网络 exit exit the program help display help hosts 主机管理功能 loot 查看嗅探到的敏感信息

Dec 30, 2022
Ping library for Golang with multi-host support

pingo Fast and lightweight ping library for Golang with multi-host support. Features ICMP sockets: UDP port 0 means "let the kernel pick a free number

Nov 9, 2022
apache dubbo gateway,L7 proxy,virtual host,k8s ingress controller.
apache dubbo gateway,L7 proxy,virtual host,k8s ingress controller.

apache dubbo gateway,L7 proxy,virtual host,k8s ingress controller.

Jul 22, 2022
This POC is built with the goal to collect events/logs from the host systems such as Kubernetes, Docker, VMs, etc. A buffering layer is added to buffer events from the collector
This POC is built with the goal to collect events/logs from the host systems such as Kubernetes, Docker, VMs, etc. A buffering layer is added to buffer events from the collector

What is does This POC is build with the goal to collect events/logs from the host systems such as Kubernetes, docker, VMs etc. A buffering layer is ad

Nov 11, 2022