Convenience wrapper around "go test" + "benchstat".

go-benchrun

Convenience wrapper around "go test" + benchstat.

Run benchmarking in 1 simple command.

Installation & Quick start

This install go-benchrun binary under your $GOPATH/bin:

go get github.com/Quasilyte/go-benchrun

If $GOPATH/bin is under your system $PATH, go-benchrun command should be available after that.
This should print the help message:

$ go-benchrun --help
Usage: go-benchrun [flags...] oldBench newBench [go test args...]
* oldBench is a pattern for `old` benchmark (w/o `Benchmark` prefix)
* newBench is a pattern for `new` benchmark (w/o `Benchmark` prefix)

Example:
	# compare BenchmarkOld and BenchmarkNew from foopkg package with -count=10
	$ go-benchrun Old New -v -count=10 foopkg

Flags and defaults:
  -newFile string
    	new benchmark results destination file (default "./new.txt")
  -oldFile string
    	old benchmark results destination file (default "./old.txt")

See "Workflow" section for more usage info".

Workflow

Without go-benchrun, your workflow is either of these two:

  1. Rely on VSC.
    • Store old benchmark results (run go test).
    • Apply optimizations.
    • Run benchmarks again with optimized code.
    • Compare results with benchstat.
    • If you need to switch between implementations, you use stash and/or branches.
  2. Rely on renaming.
    • Use one branch, two different benchmarks.
    • Collect results from both benchmarks.
    • Before running benchstat, rename benchmarks, so their name matches.

go-benchrun automates (2) scheme for you.

  1. First, it runs -bench=oldBench and saves results to oldFile.
  2. Then it runs -bench=newBench and saves results to newFile.
  3. After that, it renames newBench from newFile to oldBench.
  4. Finally, it runs benchstat -geomean oldFile newFile.

For example, lets say that you have this test file with benchmarks:

package benchmark

import (
	"testing"
)

//go:noinline
func emptySliceLit() []int {
	return []int{}
}

//go:noinline
func makeEmptySlice() []int {
	return make([]int, 0)
}

func BenchmarkEmptySliceLit(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = emptySliceLit()
	}
}

func BenchmarkMakeEmptySlice(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = makeEmptySlice()
	}
}

In order to compare BenchmarkEmptySliceLit and BenchmarkMakeEmptySlice you do:

$ go-benchrun EmptySliceLit MakeEmptySlice -v -count=5 .
  Running old benchmarks:
goos: linux
goarch: amd64
BenchmarkEmptySliceLit-8   	300000000	         5.79 ns/op
BenchmarkEmptySliceLit-8   	300000000	         5.66 ns/op
BenchmarkEmptySliceLit-8   	300000000	         5.70 ns/op
BenchmarkEmptySliceLit-8   	300000000	         5.75 ns/op
BenchmarkEmptySliceLit-8   	300000000	         5.84 ns/op
PASS
ok  	_/home/quasilyte/CODE/go/bench	11.595s
  Running new benchmarks:
goos: linux
goarch: amd64
BenchmarkMakeEmptySlice-8   	200000000	         6.77 ns/op
BenchmarkMakeEmptySlice-8   	200000000	         6.52 ns/op
BenchmarkMakeEmptySlice-8   	200000000	         6.30 ns/op
BenchmarkMakeEmptySlice-8   	300000000	         5.89 ns/op
BenchmarkMakeEmptySlice-8   	200000000	         7.03 ns/op
PASS
ok  	_/home/quasilyte/CODE/go/bench	10.375s
  Benchstat results:
name             old time/op  new time/op  delta
EmptySliceLit-8  5.75ns ± 2%  6.50ns ± 9%  +13.12%  (p=0.008 n=5+5)

To skip unit tests, specify -run flag for go test, as usual.

Owner
Iskander (Alex) Sharipov
Iskander (Alex) Sharipov
Similar Resources

Go wrapper around the Iup GUI toolset

Iup Go Wrapper iup is a Go wrapper around the Iup GUI toolkit. The project was started on April 27, 2011. Fork https://github.com/grd/iup is a fork of

Nov 28, 2020

A simple wrapper around libpcap for the Go programming language

PCAP This is a simple wrapper around libpcap for Go. Originally written by Andreas Krennmair [email protected] and only minorly touched up by Mark Smith

Sep 13, 2022

A tiny wrapper around NSQ topic and channel :rocket:

Event Bus NSQ A tiny wrapper around go-nsq topic and channel. Protect nsq calls with gobreaker. Installation go get -u github.com/rafaeljesus/nsq-even

Sep 27, 2022

Package set is a small wrapper around the official reflect package that facilitates loose type conversion and assignment into native Go types.

Package set is a small wrapper around the official reflect package that facilitates loose type conversion and assignment into native Go types. Read th

Sep 4, 2022

A wrapper around cd and xdg-open to use aliases to directory paths in terminal

CDD - Change directories Dynamically I am a lazy linux user and turns out that basic tab completion wasn't enough for me to navigate through my direct

Apr 21, 2022

a simple wrapper around resty to report HTTP calls metrics to prometheus

restyprom a simple wrapper around resty to report HTTP calls metrics to prometheus If you're using resty and want to have metrics of your HTTP calls,

Mar 16, 2022

The NVD API is an unofficial Go wrapper around the NVD API.

NVD API The NVD API is an unofficial Go wrapper around the NVD API. Supports: CVE CPE How to use The following shows how to basically use the wrapper

Jul 20, 2022

Unofficial but convenient Go wrapper around the NVD API

NVD API The NVD API is an unofficial Go wrapper around the NVD API. Supports: CVE CPE How to use The following shows how to basically use the wrapper

Nov 1, 2021

Go wrapper around Device Console Windows tool.

go-devcon Go wrapper around the Windows Device Console (devcon.exe). go install github.com/mikerourke/go-devcon Introduction Here's a brief overview

Nov 4, 2021

A Go wrapper around the Notion API

go-notion A Go wrapper around the Notion API. | ⚠ This package is new and under active development. How to Use Install the package go get github.com/b

Nov 19, 2021

Testing the use of a golang wrapper around UserMode Linux for making stdin

This code is for testing the use of a golang wrapper around UserMode Linux for making stdin, stdout and stderr available to attach, detach and reattach to from the host using Unix sockets.

Dec 24, 2021

A dead simple Go wrapper around the hidden moonarch.app API.

moonarch A dead simple Go wrapper around the hidden moonarch.app API. How-To First, get the repository: go get github.com/lazdotdigital/moonarch. moon

Nov 27, 2021

Simple wrapper around multiple fs.FS instances, recursively merging them together dynamically.

go-layerfs This is a simple wrapper around multiple fs.FS instances, recursively merging them together dynamically. If you have two directories, of wh

Aug 9, 2022

A neat wrapper around the 4chan API for content scraping.

moonarch A neat wrapper around the 4chan API for content scraping. How-To First, get the repository: go get github.com/lazdotdigital/fourscrape. fours

Nov 27, 2021

A light wrapper around R.

arr A light wrapper around R. Install go get github.com/devOpifex/arr or go install github.com/devOpifex/[email protected] Help arr -h Completion See documen

Dec 11, 2021

a wrapper around BadgerDB providing a simple API.

Carbon Cache A wrapper around BadgerDB providing a simple API. NOTE This package is provided "as is" with no guarantee. Use it at your own risk and al

Aug 28, 2022

a small wrapper around go-chi HTTP router.

sigma a small wrapper around go-chi HTTP router. NOTE: This package is provided "as is" with no guarantee. Use it at your own risk and always test it

May 14, 2022

A simple wrapper around badgerDB that can be used across multiple projects

mstore Mstore is a simple wrapper around badgerDB for platform applications that require a quick persistent cache close to the consumer. It's intended

Dec 14, 2021

The Direct Route GO SDK provides a thin wrapper around the Direct Route API for sending private transactions

Direct Route Go SDK The Direct Route GO SDK provides a thin wrapper around the D

Aug 14, 2022
A Small tool for SDWAN performance test and policy validation

sdwan-perf Sdwan-perf is based on golang and could support almost platform for performance and policy validation. SDWAN Performance Test Report +--

Sep 3, 2022
A convenience library for generating, comparing and inspecting password hashes using the scrypt KDF in Go 🔑

simple-scrypt simple-scrypt provides a convenience wrapper around Go's existing scrypt package that makes it easier to securely derive strong keys ("h

Sep 11, 2022
Additions to Go's database/sql for super fast performance and convenience. (fork of gocraft/dbr)

dbr (fork of gocraft/dbr) provides additions to Go's database/sql for super fast performance and convenience. Getting Started // create a connection (

Sep 8, 2022
Additions to Go's database/sql for super fast performance and convenience.

gocraft/dbr (database records) gocraft/dbr provides additions to Go's database/sql for super fast performance and convenience. $ go get -u github.com/

Sep 14, 2022
Convenience of containers, security of virtual machines

Convenience of containers, security of virtual machines With firebuild, you can build and deploy secure VMs directly from Dockerfiles and Docker image

Sep 23, 2022
A convenience library for generating, comparing and inspecting password hashes using the scrypt KDF in Go 🔑

simple-scrypt simple-scrypt provides a convenience wrapper around Go's existing scrypt package that makes it easier to securely derive strong keys ("h

Sep 11, 2022
A convenience provider for reading `.env` files in Terraform.

Terraform Provider dotenv This is a convenience provider for Terraform that provides a data source for reading .env files. Using the provider Below is

Sep 12, 2022
Convenience file functions for Go.

files Convenience file functions for Go. What This is a small collection of file functions to make development simpler. NewCloser() creates a deferrab

Nov 20, 2021
goxml - A thin wrapper around libxml2

golibxml golibxml is a simple wrapper for libxml. The goal is to avoid any extra magic so that a more friendly library can be written to sit on top of

Oct 25, 2020
A simple wrapper around sql.DB to help with structs. Not quite an ORM.

go-modeldb A simple wrapper around sql.DB to help with structs. Not quite an ORM. Philosophy: Don't make an ORM Example: // Setup require "modeldb" db

Nov 16, 2019