Automatically update your Go tests

autogold - automatically update your Go tests Hexops logo

Go Reference

Go CI codecov Go Report Card

autogold makes go test -update automatically update your Go tests (golden files and Go values in e.g. foo_test.go.

~5m introduction available on YouTube:

"It's 2021: you shouldn't have to update Go tests manually"

Automatic golden files

Write in a Go test:

import "github.com/hexops/autogold"
...
autogold.Equal(t, got)

go test -update will now create/update a testdata/.golden file for you automatically.

Automatic inline test updating

Write in a Go test:

want := autogold.Want("my_test", nil)
want.Equal(t, got)

go test -update will automatically update the autogold.Want("my_test", ...) call with the Go syntax for whatever value your test got (complex Go struct, slices, strings, etc.)

Diffs

Anytime your test produces a result that is unexpected, you'll get very nice diffs showing exactly what changed. It does this by converting values at runtime directly to a formatted Go AST, and using the same diffing library the Go language server uses:

--- FAIL: TestEqual (0.08s)
    autogold.go:91: mismatch (-want +got):
        --- want
        +++ got
        @@ -1 +1 @@
        +&example.Baz{Name: "Jane", Age: 31}

Subtesting

Use table-driven Go subtests? autogold.Want and go test -update will automatically find and replace the nil values for you:

func TestTime(t *testing.T) {
	testCases := []struct {
		gmt  string
		loc  string
		want autogold.Value
	}{
		{"12:31", "Europe/Zuri", autogold.Want("Europe", nil)},
		{"12:31", "America/New_York", autogold.Want("America", nil)},
		{"08:08", "Australia/Sydney", autogold.Want("Australia", nil)},
	}
	for _, tc := range testCases {
		t.Run(tc.want.Name(), func(t *testing.T) {
			loc, err := time.LoadLocation(tc.loc)
			if err != nil {
				t.Fatal("could not load location")
			}
			gmt, _ := time.Parse("15:04", tc.gmt)
			got := gmt.In(loc).Format("15:04")
			tc.want.Equal(t, got)
		})
	}
}

It works by finding the relevant autogold.Want("", ...) call below the named TestTime function, and then replacing the nil parameter (or anything that was there.)

What are golden files, when should they be used?

Golden files are used by the Go authors for testing the standard library, the gofmt tool, etc. and are a common pattern in the Go community for snapshot testing. See also "Testing with golden files in Go" - Chris Reeves

Golden files make the most sense when you'd otherwise have to write a complex multi-line string or large Go structure inline in your test, making it hard to read.

In most cases, you should prefer inline snapshots, subtest golden values, or traditional Go tests.

Custom formatting

valast is used to produce Go syntax at runtime for the Go value you provide. If the default output is not to your liking, you have options:

  • Pass a string to autogold: It will be formatted as a Go string for you in the resulting .golden file / in Go tests.
  • Use your own formatting (JSON, etc.): Make your got value of type autogold.Raw("foobar"), and it will be used as-is for .golden files (not allowed with inline tests.)
  • Exclude unexported fields: autogold.Equal(t, got, autogold.ExportedOnly())

Backwards compatibility

  • As is the case with gofmt, different Go versions may produce different formattings (although rare.)
  • Minor versions of autogold (e.g. v1.0, v1.1) may alter the formatting of .golden files, although we will be mindful of such changes.
  • Major versions of autogold (e.g. v1, v2) will be used for any major changes in output that would be difficult to review (we expect this will be rare in practice.)

Alternatives comparison

The following are alternatives to autogold, making note of the differences we found that let us to create autogold:

Owner
Hexops
Experiment everywhere
Hexops
Comments
  • Cannot find autogold.Want(

    Cannot find autogold.Want("foo", ...) when using testify suite

    minimal example code, without autogold:

    package foo_test
    
    import (
            "testing"
    
            "github.com/stretchr/testify/assert"
            "github.com/stretchr/testify/suite"
    )
    
    type newUserStartTestSuite struct {
            suite.Suite
    }
    
    type NewUserStart string
    const (
            Registration = `registration`
            EmailVerification = `email_verification`
            PhoneVerification = `phone_verification`
    )
    func (n *NewUserStart) String() string {
            return string(*n)
    }
    
    func (ts *newUserStartTestSuite) TestString() {
            var s NewUserStart
    
            s = Registration
            assert.Equal(ts.T(), s.String(), "registration")
    
            s = EmailVerification
            assert.Equal(ts.T(), s.String(), "email_verification")
    
            s = PhoneVerification
            assert.Equal(ts.T(), s.String(), "phone_verification")
    }
    
    func TestNewUserStartTestSuite(t *testing.T) {
            suite.Run(t, new(newUserStartTestSuite))
    }
    

    with autogold (the test case doesn't make sense, but this is the minimal to reproduce):

    package foo_test
    
    import (
            "testing"
            "github.com/hexops/autogold"
            "github.com/stretchr/testify/suite"
    )
    
    type newUserStartTestSuite struct {
            suite.Suite
    }
    
    type NewUserStart string
    const (
            Registration = `registration`
            EmailVerification = `email_verification`
            PhoneVerification = `phone_verification`
    )
    func (n *NewUserStart) String() string {
            return string(*n)
    }
    
    func (ts *newUserStartTestSuite) TestString() {
            var s NewUserStart
    
            s = Registration
            want := autogold.Want("reg",nil)
            want.Equal(ts.T(), s.String())
    
            s = EmailVerification
            want = autogold.Want("ev",nil)
            want.Equal(ts.T(), s.String())
    
            s = PhoneVerification
            want = autogold.Want("pv",nil)
            want.Equal(ts.T(), s.String())
    }
    
    func TestNewUserStartTestSuite(t *testing.T) {
            suite.Run(t, new(newUserStartTestSuite))
    }
    

    the error go test -v foo_test.go -update

    === RUN   TestNewUserStartTestSuite
    === RUN   TestNewUserStartTestSuite/TestString
        want.go:119: autogold: foo_test.go: could not find autogold.Want("reg", ...) function call
    --- FAIL: TestNewUserStartTestSuite (0.01s)
        --- FAIL: TestNewUserStartTestSuite/TestString (0.01s)
    FAIL
    FAIL    command-line-arguments  0.015s
    FAIL
    
  • Fix data race when -update flag is enabled

    Fix data race when -update flag is enabled

    Usage of the 'cleaned' map wasn't correctly synchronized, so reads and writes by parallel tests can trigger a data race. By synchronizing around access to the map using a lock this behaviour can be corrected

    Fixes #28

  • Print in color

    Print in color

    This highlights the output in red/green and strips the -/+ characters.

    (the -/+ in this screenshot are actually part of the testdata - the -/+ from autogold have been stripped)

    CleanShot 2021-10-29 at 22 27 38@2x

    Resolves #15

  • inline-update of autogold.Value with populated *string inside got inserts invalid Go code

    inline-update of autogold.Value with populated *string inside got inserts invalid Go code

    Hello! I've got a got-struct which contains string-pointers. Using inline-update, it adds the following lines to the struct:

    MobileCountryCode: &"xxx",
    MobileNetworkCode: &"xx",
    

    But that is not valid golang code, so it fails to build. Can this be fixed somehow? Thank you! I'm thinking maybe add autogold.String which returns a *string of the passed-in string?

    A complete repro is the following. go test -update inserts the line {name: "good", want: autogold.Want("good", &A{A: &"abc", B: "def"})},, which is no valid Go code.

    func TestGet(t *testing.T) {
    	tests := []struct {
    		name string
    		want autogold.Value
    	}{
    		{name: "good", want: autogold.Want("good", nil)},
    	}
    	for _, tt := range tests {
    		t.Run(tt.name, func(t *testing.T) {
    			got := Get()
    			tt.want.Equal(t, got)
    		})
    	}
    }
    
    type A struct {
    	A *string
    	B string
    }
    
    func Get() *A {
    	str := "abc"
    
    	return &A{
    		A: &str,
    		B: "def",
    	}
    }
    
  • Go modules require a new directory

    Go modules require a new directory "v2" for v2.0.0

    It is not possible use go get like this:

    $ go get github.com/hexops/[email protected]
    go: github.com/hexops/[email protected]: invalid version: module contains a go.mod file, so module path must match major version ("github.com/hexops/autogold/v2")
    

    Please see https://go.dev/blog/v2-go-modules

    Expected behavior is that it should be possible to use 'go get' with a version tag, just like it works with the sha:

    go get github.com/hexops/autogold@57e286ce7f6d2f9144e2755638f248be64050c47
    
  • Issue with time.Time in golden struct

    Issue with time.Time in golden struct

    This works quite nicely...but I have an issue with structs using time.Time. Namely time.Time has a bunch of private internal variables, which naturally I cannot setup.

    • Can I ignore some fields in the struct?
    • Is there some special handling for time.Time?

    The time is stable since it's coming from the database.

  • Infinite loop

    Infinite loop

    I've come across a possible bug where autogold gets stuck in an infinite loop, and I don't know why.

    Here's a simple repo that should be reproducible on your end: https://github.com/randallmlough/autogold-infinite-loop-issue

    In the ast package, uncomment the last two tests. Both those get stuck in an infinite loop.

  • want-data is partly dynamic and has to be adapated to got before equality-check

    want-data is partly dynamic and has to be adapated to got before equality-check

    I have want-data for which part of got is dynamic.

    My example is this. This is the want-data:

        - root:
            items:
            - download:
                ilias: Algos 2
                name: Algos1
                options:
                  download_folder: ""
            path: 02-ss2021
        path: %s
    

    I read this data, do want := fmt.Sprintf(wantRaw, path) and then compare with assert.Equal:

        - root:
            items:
            - download:
                ilias: Algos 2
                name: Algos1
                options:
                  download_folder: ""
            path: 02-ss2021
        path: /var/folders/rw/35q09zqj5yv5pz4wwg3yjfkm0000gn/T/TestSetup_basic278973801/001
    

    I can't see how to do this with autogold.Equal at the moment. Any ideas? I guess generics would be an easy solution but that's still a long time out. :-)

  • Version 2

    Version 2

    This integrates a number of fixes/improvements based on experience using autogold over the past few years, many from using it in large enterprise codebases.

    For details on these changes see the new CHANGELOG entry: https://github.com/hexops/autogold/blob/7079f22276779239ebe4a8ff11b67f1a487385bd/README.md#v200-not-released

    Fixes #11 Fixes #20

    • [x] By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.
  • "go get" complains about "unknown revision" for dependencies

  • go test -update doesn't work when /tmp and workdir are different devices (?)

    go test -update doesn't work when /tmp and workdir are different devices (?)

    Hi! When running go test -update on a freshly created tests I get:

    ❯ go test -update                                                                                                                                                         2021-03-16 15:19:44
    --- FAIL: Test_WorstScores_nilResult (0.00s)
        autogold.go:82: rename testdata/nil.golden /tmp/go-golden-074934610/nil.golden: invalid cross-device link
    FAIL
    exit status 1
    FAIL    <pkg name redacted>  0.006s
    
    
    service/dashboard/score on master #.‍  
    ❯                                                                                                                                                                         2021-03-16 15:19:54
    

    relevant mounts:

    tmpfs on /tmp type tmpfs (rw,nosuid,nodev,nr_inodes=409600,inode64)
    /dev/mapper/mainroot on /home type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,subvolid=258,subvol=/@home)
    

    This is weird, I assume that most people have /tmp mounted as tmpfs, so someone else should already encounter this error.

  • Fix deadlock updating inline tests

    Fix deadlock updating inline tests

    Fix #36

    Signed-off-by: Timon Wong [email protected]

    • [X] By selecting this checkbox, I agree to license my contributions to this project under the license(s) described in the LICENSE file, and I have the right to do so or have received permission to do so by an employer or client I am producing work for whom has this right.
  • Inline test update runs indefinitely

    Inline test update runs indefinitely

    For the test:

    package main
    
    import (
    	"testing"
    
    	"github.com/hexops/autogold/v2"
    )
    
    func TestInlineUpdate(t *testing.T) {
    	autogold.Expect(nil).Equal(t, "1")
    }
    

    go test fails as expected, and go test -update hangs and doesn't exit. I'm running Go 1.19 on MacOS 12.0.1 with an M1 chip and the latest v2 version of Autogold.

    I tried this initially in one of my projects and it did the same thing, so I kept trying to simplify the reproduction. Since it's so simple, I suspect I'm doing something wrong: any pointers on how I should further diagnose would be appreciated!

    Note that this is inline only: golden files work as expected (see what I did there?).

  • valast: feature Request: custom time.Time/datatype result generation and comparison

    valast: feature Request: custom time.Time/datatype result generation and comparison

    Background: saving to database and generating the result giving different diff because of timezone difference

    minimal code to reproduce:

    package foo
    import "time"
    import "github.com/hexops/autogold"
    import "testing"
    func TestFoo(t *testing.T) {
            const layout = "Jan 2, 2006 at 3:04pm (MST)"
            tm, _ := time.Parse(layout, "Feb 4, 2014 at 6:05pm (PST)")
            t2 := tm.UTC() // assuming this result queried from database from column that doesn't store timezone
    
            println(tm.UnixNano(), t2.UnixNano(), tm.UnixNano() == t2.UnixNano()) // both are inherently equal
           
            want := autogold.Want("internally equal", t2)
            want.Equal(t, tm)
    }
    

    it shows:

    1391537100000000000 1391537100000000000 true
    --- FAIL: TestFoo (0.06s)
        want.go:200: mismatch (-want +got):
            --- want
            +++ got
            @@ -1 +1,10 @@
            -time.Time{ext: 63527133900}
            \ No newline at end of file
            +time.Time{ext: 63527133900, loc: &time.Location{
            +       name: "PST",
            +       zone: []time.zone{time.zone{
            +               name: "PST",
            +       }},
            +       tx:         []time.zoneTrans{time.zoneTrans{when: -9223372036854775808}},
            +       cacheStart: -9223372036854775808,
            +       cacheEnd:   9223372036854775807,
            +       cacheZone:  &time.zone{name: "PST"},
            +}}
            \ No newline at end of file
    
    FAIL
    FAIL    command-line-arguments  0.060s
    FAIL
    

    where it inherently the same.

    1. Is there a way to customize the generated struct to be using time.Unix(sec, nanosec) instead of full struct since &time.Time{ext:123} has unexported field?
    2. Is there a way to customize the comparison of value so it would consider inherently equal time ignoring the timezone as equal?
  • valast: missing import path when inlining a value from a different package

    valast: missing import path when inlining a value from a different package

    Hey there, it seems like inline snapshots don't work when using types from other packages. In my case parser and parser_test.

    I've got the following package setup:

    parser/
      parser.go
      parser_test.go
      ast.go
    

    My AST is defined in ast.go:

    package parser
    
    type AST struct { ... }
    

    I'd like to use AST in parser_test:

    package parser_test
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, nil)
      want.Equal(t, state)
    }
    

    Unfortunately when I run go test ./parser_test.go -update, it prints the following:

    package parser_test
    
    import "parser"
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, &AST{ ... })
      want.Equal(t, state)
    }
    

    When it should be:

    package parser_test
    
    import "parser"
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, &parser.AST{ ... })
      want.Equal(t, state)
    }
    
  • valast: remove redundant type

    valast: remove redundant type

    Description

    One of the built-in Goland linter gives warning about the redundant type. It would be nice if the generated result doesn't contain redundant fields. Help the code become cleaner.

    Problem

    Redundant type 
     Inspection info: Reports redundant types in composite literals.
    For example:
        [][]int{[]int{1}, []int{2}}
    can be simplified to:
        [][]int {{1}, {2}}
    .
    

    Example in Goland

    The grey-out fields are redundant types. Screenshot_2021-01-31_17-57-23

Record and replay your HTTP interactions for fast, deterministic and accurate tests

go-vcr go-vcr simplifies testing by recording your HTTP interactions and replaying them in future runs in order to provide fast, deterministic and acc

Dec 25, 2022
A next-generation testing tool. Orion provides a powerful DSL to write and automate your acceptance tests

Orion is born to change the way we implement our acceptance tests. It takes advantage of HCL from Hashicorp t o provide a simple DSL to write the acceptance tests.

Aug 31, 2022
Terratest is a Go library that makes it easier to write automated tests for your infrastructure code.

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. It provides a variety of helper functions and patterns for common infrastructure testing tasks,

Dec 30, 2022
Automatically generate Go test boilerplate from your source code.
Automatically generate Go test boilerplate from your source code.

gotests gotests makes writing Go tests easy. It's a Golang commandline tool that generates table driven tests based on its target source files' functi

Jan 8, 2023
CLI tool to mock TCP connections. You can use it with Detox, Cypress or any other framework to automatically mock your backend or database.

Falso It is a CLI that allows you to mock requests/responses between you and any server without any configuration or previous knowledge about how it w

Sep 23, 2022
Package for comparing Go values in tests

Package for equality of Go values This package is intended to be a more powerful and safer alternative to reflect.DeepEqual for comparing whether two

Dec 29, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

Dec 30, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
Ruby on Rails like test fixtures for Go. Write tests against a real database

testfixtures Warning: this package will wipe the database data before loading the fixtures! It is supposed to be used on a test database. Please, doub

Jan 8, 2023
A simple `fs.FS` implementation to be used inside tests.

testfs A simple fs.FS which is contained in a test (using testing.TB's TempDir()) and with a few helper methods. PS: This lib only works on Go 1.16+.

Mar 3, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

Dec 30, 2022
gostub is a library to make stubbing in unit tests easy

gostub gostub is a library to make stubbing in unit tests easy. Getting started Import the following package: github.com/prashantv/gostub Click here t

Dec 28, 2022
Golang application focused on tests
Golang application focused on tests

Maceio Golang application that listens for webhook events coming from Github, runs tests previously defined in a configuration file and returns the ou

Sep 8, 2021
Robust framework for running complex workload scenarios in isolation, using Go; for integration, e2e tests, benchmarks and more! 💪

e2e Go Module providing robust framework for running complex workload scenarios in isolation, using Go and Docker. For integration, e2e tests, benchma

Jan 5, 2023
gomonkey is a library to make monkey patching in unit tests easy

gomonkey is a library to make monkey patching in unit tests easy, and the core idea of monkey patching comes from Bouke, you can read this blogpost for an explanation on how it works.

Jan 4, 2023
A simple and expressive HTTP server mocking library for end-to-end tests in Go.

mockhttp A simple and expressive HTTP server mocking library for end-to-end tests in Go. Installation go get -d github.com/americanas-go/mockhttp Exa

Dec 19, 2021
How we can run unit tests in parallel mode with failpoint injection taking effect and without injection race

This is a simple demo to show how we can run unit tests in parallel mode with failpoint injection taking effect and without injection race. The basic

Oct 31, 2021
Package has tool to generate workload for vegeta based kube-api stress tests.

Package has tool to generate workload for vegeta based kube-api stress tests.

Nov 22, 2021