Fast implementation of base58 encoding on golang.

Fast Implementation of Base58 encoding

GoDoc Go Report Card Used By

Fast implementation of base58 encoding in Go.

Base algorithm is adapted from https://github.com/trezor/trezor-crypto/blob/master/base58.c

Benchmark

  • Trivial - encoding based on big.Int (most libraries use such an implementation)
  • Fast - optimized algorithm provided by this module
BenchmarkTrivialBase58Encoding-4          123063              9568 ns/op
BenchmarkFastBase58Encoding-4             690040              1598 ns/op

BenchmarkTrivialBase58Decoding-4          275216              4301 ns/op
BenchmarkFastBase58Decoding-4            1812105               658 ns/op

Encoding - faster by 6 times

Decoding - faster by 6 times

Usage example

package main

import (
	"fmt"
	"github.com/mr-tron/base58"
)

func main() {

	encoded := "1QCaxc8hutpdZ62iKZsn1TCG3nh7uPZojq"
	num, err := base58.Decode(encoded)
	if err != nil {
		fmt.Printf("Demo %v, got error %s\n", encoded, err)	
	}
	chk := base58.Encode(num)
	if encoded == string(chk) {
		fmt.Printf ( "Successfully decoded then re-encoded %s\n", encoded )
	} 
}
Owner
Comments
  • Simplify decoding (no functional changes)

    Simplify decoding (no functional changes)

    Hi! It looks like the original implementation back in https://github.com/trezor/trezor-crypto/commit/89a7d7797b806fac left a few too many sanity checks in the source. Adjusted and simplified to appease the go compiler.

    Thank you for the excellent starting base, and the extensive test suite!

    • Before
    base58$ go test -bench=. -count=1 -benchmem ./
    goos: darwin
    goarch: amd64
    pkg: github.com/mr-tron/base58
    BenchmarkTrivialBase58Encoding-8   	  182612	      7054 ns/op	    1911 B/op	      90 allocs/op
    BenchmarkFastBase58Encoding-8      	  928771	      1245 ns/op	     144 B/op	       2 allocs/op
    BenchmarkTrivialBase58Decoding-8   	  359553	      3370 ns/op	     513 B/op	      48 allocs/op
    BenchmarkFastBase58Decoding-8      	 1558393	       838 ns/op	     368 B/op	       3 allocs/op
    PASS
    ok  	github.com/mr-tron/base58	58.666s
    
    • After ( when including https://github.com/mr-tron/base58/pull/15 )
    base58$ go test -bench=. -count=1 -benchmem ./
    goos: darwin
    goarch: amd64
    pkg: github.com/mr-tron/base58
    BenchmarkTrivialBase58Encoding-8   	  155588	      7466 ns/op	    1911 B/op	      90 allocs/op
    BenchmarkFastBase58Encoding-8      	  966726	      1245 ns/op	      96 B/op	       2 allocs/op
    BenchmarkTrivialBase58Decoding-8   	  350648	      3447 ns/op	     513 B/op	      48 allocs/op
    BenchmarkFastBase58Decoding-8      	 2276010	       523 ns/op	     127 B/op	       2 allocs/op
    PASS
    ok  	github.com/mr-tron/base58	57.326s
    

    /cc @prusnak @Stebalien

  • Bug

    Bug

    	in := "1111111111"
    	fmt.Println("in: ", in)
    	v, err := base58.Decode(in)
    	fmt.Println("decode:", v, err)
    
    	fmt.Println("encode again:", base58.Encode(v))
    

    Try this with your function names instead (by the way, I suggest you get rid of Trivial* and rename the fast versions to Encode and Decode). Output should look like this:

    in:  1111111111
    decode: [0 0 0] <nil>
    encode again: 111
    
  • Simplify encoding (no functional changes)

    Simplify encoding (no functional changes)

    Reduce work-buffer alocation size to the absolute minimum. Unlike my other PR this does not change the performance at all ( it stays virtually the same ). Rather the allocation is halved, specifically https://github.com/ribasushi/base58/commit/505011d34545eb060ec69be9ab0e14c19db6e48b#diff-20a30baf90cfb9bb539a06cf6ecebcd1L36-R41

    Cheers!

  • bug fix for test

    bug fix for test

    In base58/base58_test.go line 22 make a byte array to use as random data container. But in the whole loop, data is never change and point to same slice, but in line 25 data is assigned to a new testValues. So all the testValues's field dec point to the same slice. So all the encode benchmark use a same slice in their whole loop. I think this does not match the intended purpose.

    https://github.com/mr-tron/base58/blob/89529c6904fcd077434931b4eac8b4b2f0991baf/base58_test.go#L22

  • Why '138'

    Why '138'

    https://github.com/mr-tron/base58/blob/c1bdf7c52f59d6685ca597b9955a443ff95eeee6/base58/base58.go#L42

    Hi,

    Could you please elaborate where the number '138' comes from here?

  • Move the tests and benchmarks to use the stdlib tests and benchmarks

    Move the tests and benchmarks to use the stdlib tests and benchmarks

    Move the test and benchmarks out of main.go and intomain_test.go so that the standard go tools can be used to run them. It would bring the project inline with other go projects and allow main to run code that uses base58 encoding and not just tests.

  • reduce from 3 to 2 allocations

    reduce from 3 to 2 allocations

    This patch allocates a single large intermediate buffer instead of two small ones. This doesn't make a difference in terms of benchmarks (on my machine) but lightens the load on the garbage collector a bit.

    Technically, with some unsafe code, we could get to 1. However, that's probably not worth it.

  • Performance is non-linear, thus. very bad as input gets large

    Performance is non-linear, thus. very bad as input gets large

    % dd if=/dev/random of=abcd10 bs=1 count=10000
    10000+0 records in
    10000+0 records out
    10000 bytes transferred in 0.051649 secs (193615 bytes/sec)
    % dd if=/dev/random of=abcd100 bs=1 count=100000
    100000+0 records in
    100000+0 records out
    100000 bytes transferred in 0.423879 secs (235916 bytes/sec)
    % dd if=/dev/random of=abcd1000 bs=1 count=1000000
    1000000+0 records in
    1000000+0 records out
    1000000 bytes transferred in 4.197053 secs (238262 bytes/sec)
    % time ./base58 -i abcd10 | wc -c
       13658
    ./base58 -i abcd10  0.12s user 0.01s system 93% cpu 0.132 total
    % time ./base58 -i abcd100 | wc -c
      136567
    ./base58 -i abcd100  11.44s user 0.17s system 95% cpu 12.184 total
    % time ./base58 -i abcd1000 | wc -c
     1365659
    ./base58 -i abcd1000  1298.50s user 20.66s system 93% cpu 23:33.18 total
    
msgpack.org[Go] MessagePack encoding for Golang

MessagePack encoding for Golang ❤️ Uptrace.dev - All-in-one tool to optimize performance and monitor errors & logs Join Discord to ask questions. Docu

Dec 28, 2022
Some Golang types based on builtin. Implements interfaces Value / Scan and MarshalJSON / UnmarshalJSON for simple working with database NULL-values and Base64 encoding / decoding.

gotypes Some simple types based on builtin Golang types that implement interfaces for working with DB (Scan / Value) and JSON (Marshal / Unmarshal). N

Feb 12, 2022
A high-performance 100% compatible drop-in replacement of "encoding/json"
A high-performance 100% compatible drop-in replacement of

A high-performance 100% compatible drop-in replacement of "encoding/json" You can also use thrift like JSON using thrift-iterator Benchmark Source cod

Jan 7, 2023
Faster base64 encoding for Go

base64 Faster base64 encoding for Go, based on Turbo-Base64. Features Drop-in replacement of encoding/base64. except for error messages and ignoring \

Nov 17, 2022
GED - Global-purpose Encoding / Decoding library

GED - Global-purpose Encoding / Decoding library This library lets you use common encoding/decoding schemes and allows you to define custom ones. Use

Nov 28, 2021
CBOR RFC 7049 (Go/Golang) - safe & fast with standard API + toarray & keyasint, CBOR tags, float64/32/16, fuzz tested.
CBOR RFC 7049 (Go/Golang) - safe & fast with standard API + toarray & keyasint, CBOR tags, float64/32/16, fuzz tested.

CBOR library in Go fxamacker/cbor is a CBOR encoder & decoder in Go. It has a standard API, CBOR tags, options for duplicate map keys, float64→32→16,

Jan 6, 2023
csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.
csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

csvutil Package csvutil provides fast and idiomatic mapping between CSV and Go (golang) values. This package does not provide a CSV parser itself, it

Jan 6, 2023
A go library that facilitates the implementation of decorator pattern.

go-decorator go-decorator is a library that facilitates the implementation of decorator pattern. Installation To install go-decorator, use go get: go

Nov 25, 2021
auto-generate capnproto schema from your golang source files. Depends on go-capnproto-1.0 at https://github.com/glycerine/go-capnproto

bambam: auto-generate capnproto schema from your golang source files. Adding capnproto serialization to an existing Go project used to mean writing a

Sep 27, 2022
Golang binary decoder for mapping data into the structure

binstruct Golang binary decoder to structure Install go get -u github.com/ghostiam/binstruct Examples ZIP decoder PNG decoder Use For struct From file

Dec 17, 2022
Fixed width file parser (encoder/decoder) in GO (golang)

Fixed width file parser (encoder/decoder) for GO (golang) This library is using to parse fixed-width table data like: Name Address

Sep 27, 2022
Encode and decode Go (golang) struct types via protocol buffers.

protostructure protostructure is a Go library for encoding and decoding a struct type over the wire. This library is useful when you want to send arbi

Nov 15, 2022
golang struct 或其他对象向 []byte 的序列化或反序列化

bytecodec 字节流编解码 这个库实现 struct 或其他对象向 []byte 的序列化或反序列化 可以帮助你在编写 tcp 服务,或者需要操作字节流时,简化数据的组包、解包 这个库的组织逻辑 copy 借鉴了标准库 encoding/json ?? 安装 使用 go get 安装最新版本

Jun 30, 2022
Dynamically Generates Ysoserial's Payload by Golang
Dynamically Generates Ysoserial's Payload by Golang

Gososerial 介绍 ysoserial是java反序列化安全方面著名的工具 无需java环境,无需下载ysoserial.jar文件 输入命令直接获得payload,方便编写安全工具 目前已支持CC1-CC7,K1-K4和CB1链 Introduce Ysoserial is a well-

Jul 10, 2022
A k-mer serialization package for Golang
A k-mer serialization package for Golang

.uniq v5 This package provides k-mer serialization methods for the package kmers, TaxIds of k-mers are optionally saved, while there's no frequency in

Aug 19, 2022
gogoprotobuf is a fork of golang/protobuf with extra code generation features.

GoGo Protobuf looking for new ownership Protocol Buffers for Go with Gadgets gogoprotobuf is a fork of golang/protobuf with extra code generation feat

Nov 26, 2021
generic sort for slices in golang

slices generic sort for slices in golang basic API func BinarySearch[E constraints.Ordered](list []E, x E) int func IsSorted[E constraints.Ordered](li

Nov 3, 2022
Bitbank-trezor - Bitbank trezor with golang

Bitbank - Trezor (c) 2022 Bernd Fix [email protected] >Y< bitbank-trezor is fre

Jan 27, 2022
Base58 encoding for Go

Base58 encoding for Go Base58 is a less known encoding, but it is popular when embedding resource identifiers into URLs. The encoded data contains onl

Dec 20, 2021
A distributed unique ID generator of using Sonyflake and encoded by Base58

Indigo About A distributed unique ID generator of using Sonyflake and encoded by Base58. ID max length is 11 characters by unsigned int64 max value. A

Nov 24, 2022