Rosetta-ergo provides a reference implementation of the Rosetta API for Ergo in Golang

Rosetta

Rosetta Ergo

Overview

rosetta-ergo provides a reference implementation of the Rosetta API for Ergo in Golang. If you haven't heard of the Rosetta API, you can find more information here.

Terminology

Rosetta uses abstract terminology to represent entities in a blockchain.

Here's how they relate to ergo:

Rosetta Ergo Notes
coin utxo, ergo box
account address Any ergo address type

Implementation status

Data

  • Network
    • /network/list
    • /network/options
    • /network/status
  • Account
    • /account/balance
    • /account/coins
  • Block
    • /block
    • /block/transaction
  • Mempool
    • /mempool
    • /mempool/transaction

Construction

  • Construction
    • /construction/combine
    • /construction/derive
    • /construction/hash
    • /construction/metadata
    • /construction/parse
    • /construction/payloads
    • /construction/preprocess
    • /construction/submit
Comments
  • enable badger truncate to fix db corruptions

    enable badger truncate to fix db corruptions

    Fixes #16

    • Fixes corruption when process is abruptly shutdown (power outage etc)
    • Also fixes the issue in the linked issue where db is corrupted after nil pointer error

    Resources:

    • https://github.com/dgraph-io/badger/issues/434
    • https://github.com/dgraph-io/badger/issues/744
    • https://github.com/dgraph-io/badger/issues/476
  • Database corruption

    Database corruption

    Sometimes when shutting down the database is corrupted.

    It's proving to be hard to recreate but when I saw this occur the stack trace in the previous execution of the app indicated a nil pointer error in converter.BlockToRosettaBlock.

    The nil pointer error doesn't seem to occur during normal execution.

    2022-03-17T06:30:33.380+1100   DEBUG   indexer indexer/indexer.go:415  block seen      {"hash": "098af34d932f123
    ce412d8640e3fa2b31b88e4c973def7fda16d01d4804b2c0f", "index": 1705}
    2022-03-17T06:30:33.381+1100    DEBUG   indexer indexer/indexer.go:306  block added     {"hash": "098af34d932f123ce412d8640e3fa2b31b88e4c973def7fda16d01d4804b2c0
    f", "index": 1705, "transactions": 1, "ops": 3}
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff6b1603646]
    
    goroutine 60 [running]:
    github.com/ross-weir/rosetta-ergo/pkg/rosetta.(*BlockConverter).BlockToRosettaBlock(0xc00007ede0, {0x7ff6b1c457b8, 0xc00007c900}, 0x0)
            V:/Development/rosetta-ergo/pkg/rosetta/converter.go:43 +0x46
    github.com/ross-weir/rosetta-ergo/pkg/indexer.(*Indexer).Block(0xc00007ede0, {0x7ff6b1c457b8, 0xc00007c900}, 0xc006a8a200, 0xc006ba4370)
            V:/Development/rosetta-ergo/pkg/indexer/indexer.go:208 +0x2f0
    github.com/coinbase/rosetta-sdk-go/syncer.(*Syncer).fetchBlockResult(0xc000593d40, {0x7ff6b1c457b8, 0xc00007c900}, 0xc006878cf0, 0x6aa)
            C:/Users/Ross/go/pkg/mod/github.com/coinbase/[email protected]/syncer/syncer.go:252 +0xae
    github.com/coinbase/rosetta-sdk-go/syncer.(*Syncer).processBlocks(0xc000593d40, {0x7ff6b1c457b8, 0xc00007c900}, 0x20, 0xacc8d)
            C:/Users/Ross/go/pkg/mod/github.com/coinbase/[email protected]/syncer/syncer.go:355 +0xc5
    github.com/coinbase/rosetta-sdk-go/syncer.(*Syncer).sequenceBlocks(0xc000593d40, {0x7ff6b1c457b8, 0xc00007c900}, {0x7ff6b1c457b8, 0xc0059f4a00}, 0xc00015a580, 0x
    c000035140, 0xc0000351a0, 0x0)
            C:/Users/Ross/go/pkg/mod/github.com/coinbase/[email protected]/syncer/syncer.go:485 +0x16f
    github.com/coinbase/rosetta-sdk-go/syncer.(*Syncer).syncRange(0xc000593d40, {0x7ff6b1c457b8, 0xc00007c900}, 0xacc8d)
            C:/Users/Ross/go/pkg/mod/github.com/coinbase/[email protected]/syncer/syncer.go:584 +0x35b
    github.com/coinbase/rosetta-sdk-go/syncer.(*Syncer).Sync(0xc000593d40, {0x7ff6b1c457b8, 0xc00007c900}, 0x695, 0xffffffffffffffff)
            C:/Users/Ross/go/pkg/mod/github.com/coinbase/[email protected]/syncer/syncer.go:649 +0x1f3
    github.com/ross-weir/rosetta-ergo/pkg/indexer.(*Indexer).Sync(0xc00007ede0, {0x7ff6b1c457b8, 0xc00007c900})
            V:/Development/rosetta-ergo/pkg/indexer/indexer.go:162 +0x446
    github.com/ross-weir/rosetta-ergo/cmd.startOnlineDependencies.func1()
            V:/Development/rosetta-ergo/cmd/run.go:100 +0x25
    golang.org/x/sync/errgroup.(*Group).Go.func1()
            C:/Users/Ross/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x67
    created by golang.org/x/sync/errgroup.(*Group).Go
            C:/Users/Ross/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x92
    
    
  • fix account balances

    fix account balances

    fixes #12

    Not sure that the processing block multiple times is actually a bug or is intended by the coinbase syncer package, the positive value inputs is certainly a bug though, didn't track down the root cause where these inputs are actually originating from.

  • Incorrect account balances - Some blocks processed multiple times

    Incorrect account balances - Some blocks processed multiple times

    • indexer.BlockAdded called multiple times for the same block
    • Subsequent calls seem to have positive value for operation inputs leading to incorrect account balances

    Rosetta conversion/parsing looks good, something is causing blocks to be processed more than once and on subsequent tries the inputs do not have negated values, the coins must be coming from somewhere that isn't the rosetta ergo converter or there's a case where supplied inputs in the converter aren't negated.

    It's also possible it is being passed through the converter but we're negating an already negated value.

  • Refactor codebase

    Refactor codebase

    • Extract storage from indexer so it can be reused
    • Remove dependency on indexer from services
    • Structure the codebase using pkg convention to make distinction between code and non-code directories in root

    TODO:

    • [x] Remove conversion/knowledge of rosetta from the ergo client package and into a converter package
    • [x] Use conversion functions in services to perform rosetta <-> ergo type conversion

    Addresses first todo in #7

  • Change how we determine the node is ready

    Change how we determine the node is ready

    At the moment we poll until the node is able to serve the genesis block, this has sometimes caused issues where the indexer catches the node height and then fails to process blocks because certain blocks don't exist in the node yet.

    Instead we should wait until node fullHeight is within n threshold of node headersHeight, or even just wait until they're equal to be safe.

  • Can't process negative balance on startup

    Can't process negative balance on startup

    022/03/17 11:18:09 Syncing 529641-707872
    2022-03-17T11:18:09.328+1100    DEBUG   indexer indexer/indexer.go:415  block seen      {"hash": "494a2cfc3ad748d6a38cb0967e1de77cc591addb167e213afd6750bbb29aca05", "index": 529641}
    2022-03-17T11:18:09.351+1100    INFO    storage storage/storage.go:97   database closed successfully
    2022-03-17T11:18:09.351+1100    ERROR   main    [email protected]/sugar.go:191        Ignored key without a value.    {"ignored": "unable to process blocks: unable to process block: negative balance -294582075:&{Symbol:ERG Decimals:9 Metadata:map[]} for &{Address:9
    f39hRYx8Txmcwqbh4KSuLUCEfSezYp1kVJggj7Z4ZTz1zG2RhS SubAccount:<nil> Metadata:map[]} at &{Index:529641 Hash:494a2cfc3ad748d6a38cb0967e1de77cc591addb167e213afd6750bbb29aca05}: unable to add block to storage 494a2cfc3ad748d6a38cb0967e1de77cc591addb167e213afd6750
    bbb29aca05:529641: unable to sync to 707872"}
    

    Happens sometimes after shutting down the indexer, seems to happen at higher block heights. Maybe something to do with the size of the blocks that are cached and restored at start up.

  • Fix release pipeline

    Fix release pipeline

    Cross compiling to multiple platforms is a bit trickier with CGO (required for ztsd, likely also for ergo-lib in the future), will need separate goreleaser builds with specific CC settings, etc

    Resources:

    • https://github.com/goreleaser/goreleaser-cross-example
    • https://github.com/goreleaser/goreleaser-action/issues/233
    • https://github.com/mattn/go-sqlite3/issues/303

    Can we do something like here:

    • https://github.com/karalabe/xgo/blob/master/docker/go-1.4/Dockerfile
    • https://github.com/karalabe/xgo

    Might be able to exclude zstd with a go build tag: -tags

  • Refactor/tidy codebase

    Refactor/tidy codebase

    Now that we have some implementation verification coverage via rosetta-cli complete the following tasks before starting on construction API:

    • [x] Refactor/tidy codebase, things are a mess right now
    • [ ] Add unit tests
    • [ ] Add rosetta-cli tests to CI
    • [ ] Create docker image with ergo node & rosetta
Merchant API reference implementation

mAPI More details available in the BRFC Spec for Merchant API. The old golang (v1.1) implementation is no longer being maintained and has been moved t

Dec 14, 2022
Kava - Reference implementation of Kava, a blockchain for cross-chain DeFi. Built using the cosmos-sdk

DeFi for Crypto. Telegram | Medium | Discord Reference implementation of Kava, a

Apr 4, 2022
Go language implementation of a blockchain based on the BDLS BFT protocol. The implementation was adapted from Ethereum and Sperax implementation

BDLS protocol based PoS Blockchain Most functionalities of this client is similar to the Ethereum golang implementation. If you do not find your quest

Oct 14, 2022
Go-block-api - Golang implementation of Ethereum Block API

Go Ethereum Block API Golang implementation of Ethereum Block API This API can s

Jan 13, 2022
Pure Go implementation of the NaCL set of API's

go-nacl This is a pure Go implementation of the API's available in NaCL: https://nacl.cr.yp.to. Compared with the implementation in golang.org/x/crypt

Dec 16, 2022
Package pinentry provides a client to GnuPG's pinentry.

Package pinentry provides a client to GnuPG's pinentry.

Dec 9, 2022
Supports the safe and convenient execution of asynchronous computations with goroutines and provides facilities for the safe retrieval of the computation results.

Rendezvous The Rendezvous library supports the safe and convenient execution of asynchronous computations with goroutines and provides facilities for

Dec 29, 2021
A simplified blockchain implementation in Golang

A simplified blockchain implementation in Golang

Dec 31, 2022
Interblockchain communication protocol (IBC) implementation in Golang.

ibc-go Interblockchain communication protocol (IBC) implementation in Golang built as a SDK module. Components Core The core/ directory contains the S

Jan 7, 2023
Celer cBridge relay node implementation in Golang

cBridge Relay Node Official implementation of cBridge relay node in Golang. Prerequisites Prepare Machine To run a cBridge relay node, it is recommend

Sep 27, 2022
Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated builds are available for stable releases and the unstable master branch

Nov 24, 2021
Golang interface for local/remote DRM CDM services (NO DRM IMPLEMENTATION HERE)

NO DRM IMPLEMENTATION HERE! ONLY ABSTRAT INTERFACE! What It's a generalized interface for different types of CDM for WEBDL use. A remote CDM JSON-RPC

Oct 24, 2022
Implementation of blockchain using golang

go-blockchain Basic implementation of blockchain using golang. Initialize Blockc

Feb 18, 2022
Mini Blockchain Implementation In Golang Inspired by Go-Ethereum🚀

JP Blockchain ?? ?? Mini Blockchain Implementation In Golang Inspired by Go Ethereum & BlockChain Bar by Lukas (Web3Coach) Features Working Core Compo

Aug 8, 2022
Vso-hash - Golang implementation of the BuildXL paged hash function

vso-hash Golang implementation of the BuildXL paged hash function See https://gi

Jan 3, 2022
RepoETH - Official Golang implementation of the Ethereum protocol
RepoETH - Official Golang implementation of the Ethereum protocol

HANNAGAN ALEXANDRE Powershell Go Ethereum Official Golang implementation of the

Jan 3, 2022
Go-ethereum - Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated b

Jan 4, 2022
Btcix - Golang implementation for BTCIX Network

BTCIX Mainnet client Golang implementation for BTCIX Network Mainnet information

Dec 6, 2022
Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated builds are available for stable releases and the unstable master branch

Sep 20, 2022