A collection of packages to augment the go testing package and support common patterns.

gotest.tools

A collection of packages to augment testing and support common patterns.

GoDoc CircleCI Go Reportcard

Usage

With Go modules enabled (go1.11+)

$ go get gotest.tools/v3
import "gotest.tools/v3/assert"

To use gotest.tools with an older version of Go that does not understand Go module paths pin to version v2.3.0.

Packages

  • assert - compare values and fail the test when a comparison fails
  • env - test code which uses environment variables
  • fs - create temporary files and compare a filesystem tree to an expected value
  • golden - compare large multi-line strings against values frozen in golden files
  • icmd - execute binaries and test the output
  • poll - test asynchronous code by polling until a desired state is reached
  • skip - skip a test and print the source code of the condition used to skip the test

Related

Contributing

See CONTRIBUTING.md.

Comments
  • unable to compile tests due to

    unable to compile tests due to "expects import"

    I am not sure what happened, but the tests worked when I pushed them to our repository, now when I try to run go test I get these errors (changed directory names):

    $ go test
    # ~/git/repo
    package ~/git/repo (test)
    	imports gotest.tools/assert
    	imports gotest.tools/v3/assert/cmp: code in directory ~/go/src/gotest.tools/assert/cmp expects import "gotest.tools/assert/cmp"
    FAIL	~/git/repo [setup failed]
    

    This is how I import gotest.tools:

    import (
    	"gotest.tools/assert"
    	"net/http"
    	"testing"
    )
    

    How does this happen all of a sudden? I tried reinstalling everything, even cleared my GOPATH completely, but the error remains.

  • Errors from Golden could provide a hint about updating

    Errors from Golden could provide a hint about updating

    Passing -test.update-golden to update the golden files works well, but if you're not already familiar with the library, then it's not especially discoverable.

    Could the assertion failure include some help text which tells the user that this flag exists?

  • TestDeepEqualFailure fails w/ go 1.12.2

    TestDeepEqualFailure fails w/ go 1.12.2

    gotest.tools 2.3.0 w/ Go 1.12.2 on Fedora Rawhide, x86_64:

    Testing    in: /builddir/build/BUILD/gotest.tools-2.3.0/_build/src
             PATH: /builddir/build/BUILD/gotest.tools-2.3.0/_build/bin:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
           GOPATH: /builddir/build/BUILD/gotest.tools-2.3.0/_build:/usr/share/gocode
      GO111MODULE: off
          command: go test -buildmode pie -compiler gc -ldflags "-X gotest.tools/version=2.3.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
          testing: gotest.tools
    gotest.tools/assert
    --- FAIL: TestDeepEqualFailure (0.00s)
        assert_test.go:305: should have failure message "assertion failed: \n--- actual\n+++ expected\n{assert.stub}.b:\n\t-: 1\n\t+: 2\n", got "assertion failed: \n--- actual\n+++ expected\n\u00a0\u00a0assert.stub{\n\u00a0\u00a0\ta: \"ok\",\n-\u00a0\tb: 1,\n+\u00a0\tb: 2,\n\u00a0\u00a0}\n"
    FAIL
    exit status 1
    FAIL	gotest.tools/assert	0.028s
    
  • Add MatchExtraFilesGlob PathOp

    Add MatchExtraFilesGlob PathOp

    This PR adds MatchExtraFilesGlob PathOp. As discuss in #133, this is a starting point.

    @dnephin regarding the function

     func MatchExtraFilesGlob(glob string, ops ...PathOp) PathOp 
    

    What should I do with the ops parameter? should I apply it to the directoryPath in the AddGlob function (as it is done for other directoryPath functions)?

    Signed-off-by: glefloch [email protected]

  • Upgrade to go-cmp v0.4.0

    Upgrade to go-cmp v0.4.0

    and adjust assert/cmp/compare_test.go accordingly.

    In go-cmp v0.4.0, for user convenience, full name of the type is printed in the panic message when accessing an unexported field, see google/go-cmp#171

    Many thanks!

    Anthony

  • icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    this is a follow-up to https://github.com/gotestyourself/gotest.tools/pull/217, opening as draft because the first 3 commits are from that pull request; I thought it was good to open this change as a separate PR for visibility.

    go.mod: golang.org/x/tools v0.1.0

    Project now started to do releases; this one was related to a security issue on Windows, replacing all usages of os/exec with golang.org/x/sys/execabs.

    icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    Following the changes in Go, and golang.org/x/tools themselves, this change ensures that packages using exec.LookPath or exec.Command to find or run binaries do not accidentally run programs from the current directory when they mean to run programs from the system PATH instead.

  • go.mod: update dependencies

    go.mod: update dependencies

    go.mod: github.com/pkg/errors v0.9.1

    This version provides compatibility with Go 1.3+ native error wrapping

    go.mod: github.com/google/go-cmp v0.5.5

    Version v0.5.x provides various improvements to the reporter output; https://github.com/google/go-cmp/releases

    full diff: https://github.com/google/go-cmp/compare/v0.4.0...v0.5.5

  • Typed nil errors should not pass

    Typed nil errors should not pass "NilError"

    I don't think the fix for #167 (#171) was entirely correct. A typed nil error is not a nil error when compared against nil in a normal context.

    package baderr
    
    import (
    	"testing"
    
    	"gotest.tools/v3/assert"
    )
    
    type myEvilError struct{}
    
    func (m *myEvilError) Error() string {
    	return "this is an error"
    }
    
    func doSomething() error {
    	var err *myEvilError
    	return err // NOT nil, this has a type.
    }
    
    // Should not pass.
    func TestErrorNil(t *testing.T) {
    	assert.NilError(t, doSomething()) // This says it's nil.
    }
    
    func TestErrorNotNil(t *testing.T) {
    	assert.Assert(t, doSomething() != nil) // An actual comparison says it's not nil.
    }
    
  • Update import alias for major version

    Update import alias for major version

    In order to make vendoring work with go modules we need the alias to be as the major version, otherwise, the vendor files are in gotest.tools/v3/assert but the import says gotest.tools/assert which makes go fail to build without go modules.

    Takes over #162 Closes #162

    Signed-off-by: Vincent Demeester [email protected]

  • Issue when checking for nil custom error

    Issue when checking for nil custom error

    Create a custom error by implementing the error interface (e.g. Error() function)

    type MyCustomError struct {
        msg string
    }
    func (ce *MyCustomError) Error() string {
        return ce.string
    }
    

    In a test, checking for a nil MyCustomError reference using the NilError function will lead to unexpected behavior, such as:

    var myErr *MyCustomError 
    assert.NilError(t, myErr)
    

    in this case, you receive a memory address error, because the above custom Error() function will get invoked, even though ce is nil.

    If you "fix" your Error() function, such as:

    func (ce *MyCustomError) Error() string {
        if ce != nil {
            return ce.string
        }
        return ""
    }
    

    then you get rid of the null pointer memory error, but the assertion will fail.

    This happens because, in the assert.go file, at the type switch switch check := comparison.(type), it resolves to case error branch.

    Workaround: use Assert(t, myErr == nil) instead.

  • Go 1.11: example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args

    Go 1.11: example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args

    The poll_test unit test is calling a broken example file

    + LDFLAGS='-X gotest.tools/version=2.1.0'
    + GO_TEST_FLAGS='-buildmode pie -compiler gc'
    + GO_TEST_EXT_LD_FLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '
    + go-rpm-integration check -i gotest.tools -b /builddir/build/BUILD/gotest.tools-2.1.0/_build/bin -s /builddir/build/BUILD/gotest.tools-2.1.0/_build -V 2.1.0-1.0.fc30.llt -p /builddir/build/BUILDROOT/golang-gotest-tools-2.1.0-1.0.fc30.llt.x86_64 -g /usr/share/gocode -r '.*example.*'
    Testing in: /builddir/build/BUILD/gotest.tools-2.1.0/_build/src
          PATH: /builddir/build/BUILD/gotest.tools-2.1.0/_build/bin:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
        GOPATH: /builddir/build/BUILD/gotest.tools-2.1.0/_build:/usr/share/gocode
       command: go test -buildmode pie -compiler gc -ldflags "-X gotest.tools/version=2.1.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
       testing: gotest.tools
    gotest.tools/assert
    PASS
    ok  	gotest.tools/assert	0.028s
    gotest.tools/assert/cmd/gty-migrate-from-testify
    PASS
    ok  	gotest.tools/assert/cmd/gty-migrate-from-testify	5.330s
    gotest.tools/assert/cmp
    PASS
    ok  	gotest.tools/assert/cmp	0.009s
    gotest.tools/assert/opt
    PASS
    ok  	gotest.tools/assert/opt	0.008s
    gotest.tools/env
    PASS
    ok  	gotest.tools/env	0.008s
    gotest.tools/fs
    PASS
    ok  	gotest.tools/fs	0.014s
    gotest.tools/golden
    PASS
    ok  	gotest.tools/golden	0.009s
    gotest.tools/icmd
    PASS
    ok  	gotest.tools/icmd	0.369s
    gotest.tools/poll
    # gotest.tools/poll_test
    ./example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args
    FAIL	gotest.tools/poll [build failed]
    
  • Generics in comparison functions?

    Generics in comparison functions?

    Hello!

    I was wondering if you'd considered whether it's worth it to use generics in the comparison functions, and what a migration path to that model might look like?

  • Feature request: Add a flag for updating golden files interactively

    Feature request: Add a flag for updating golden files interactively

    It would be nice if there was a way to run the tests, and for each one that fails, prompt the user to decide whether to update the diff or not. Think a UX similar to git add -p

  • Allow comparing zero for DurationWithThreshold

    Allow comparing zero for DurationWithThreshold

    I found DurationWithThreshold returns false if both of the values are zero.

    https://github.com/gotestyourself/gotest.tools/pull/62/files#diff-d952cf5be65334da619358823c8ff4669961b0d9a30e85d542caad125beac4c6R20

    I guess if both of them are zero, we should return true to compare all test cases in the consistent way.

    What do you think?

  • skip: deprecate skip.If and replace it with assert.SkipIf

    skip: deprecate skip.If and replace it with assert.SkipIf

    Originally we thought there might be more stuff to include in the skip package. It has been a few years, and we haven't added anything.

    This PR deprecates the skip package, and moves the one function to assert.SkipIf. Instead of a very small page it should be fine to move this one exported function to the assert package.

    Also improves the godoc a bit, and documents msgAndArgs on assert.Assert as well.

  • Helper for limiting a test's duration?

    Helper for limiting a test's duration?

    I've run across a few cases recently where I've wanted to set a deadline on a test running. Generally when doing potentially complicated things that could deadlock, and as mistake could accidentally leave the test hanging.

    Would you be interesting in accepting a new function/package for this functionality?

    There's some related disucssion in https://github.com/golang/go/issues/10529 where it was deemed not suitable for core.

  • Allow specifying default deep-equality comparison options?

    Allow specifying default deep-equality comparison options?

    I originally tried pitching this to go-cmp, and they said no because they quite resonably want all comparison to be stateless.

    https://github.com/google/go-cmp/issues/241#issue-736205887

    The context from that issue is:

    After upgrading to the most recent version of protobuf, we are finding all of our comparisons are failing due to some new unexported fields.

    There is a cmp.Option implementation provided by the protobuf module in godoc.org/google.golang.org/protobuf/testing/protocmp#Transform - but I now need to go and add this option to every single compare.

    This particular check is generic and safe to add on every compare - would it be reasonable to provide a way to install such global comparison options?

    This feels like it might be slightly safer for a test assertion tool, as each "global" would be scoped to the test binary.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Dec 12, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Nov 24, 2022
A presentable test report generator for go packages
A presentable test report generator for go packages

go-tprof Overview Prerequisites 1. Go version >= 1.12 2. Node.js version >= 8.0.0 (for building the UI) 3. Yarn 4. GOPATH and local bin setup (`expor

Dec 16, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022
A toolkit with common assertions and mocks that plays nicely with the standard library

Testify - Thou Shalt Write Tests ℹ️ We are working on testify v2 and would love to hear what you'd like to see in it, have your say here: https://cutt

Dec 30, 2022
A mock of Go's net package for unit/integration testing

netmock: Simulate Go network connections netmock is a Go package for simulating net connections, including delays and disconnects. This is work in pro

Oct 27, 2021
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Jan 2, 2023
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
Minimal and Beautiful Go testing framework
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

Dec 25, 2022
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Jan 5, 2023
HTTP traffic mocking and testing made easy in Go ༼ʘ̚ل͜ʘ̚༽

gock Versatile HTTP mocking made easy in Go that works with any net/http based stdlib implementation. Heavily inspired by nock. There is also its Pyth

Jan 4, 2023
HTTP mock for Golang: record and replay HTTP/HTTPS interactions for offline testing

govcr A Word Of Warning I'm in the process of partly rewriting govcr to offer better support for cassette mutations. This is necessary because when I

Dec 28, 2022
A next-generation testing tool. Orion provides a powerful DSL to write and automate your acceptance tests

Orion is born to change the way we implement our acceptance tests. It takes advantage of HCL from Hashicorp t o provide a simple DSL to write the acceptance tests.

Aug 31, 2022
Quick and Easy server testing/validation
Quick and Easy server testing/validation

Goss - Quick and Easy server validation Goss in 45 seconds Note: For an even faster way of doing this, see: autoadd Note: For testing docker container

Oct 7, 2022
Stress testing and benchmarking tool for the NEAR EVM

evm-bully --- stress testing and benchmarking tool for the NEAR EVM

May 30, 2022
An always-on framework that performs end-to-end functional network testing for reachability, latency, and packet loss

Arachne Arachne is a packet loss detection system and an underperforming path detection system. It provides fast and easy active end-to-end functional

Dec 31, 2022
HTTP load testing tool and library. It's over 9000!
HTTP load testing tool and library. It's over 9000!

Vegeta Vegeta is a versatile HTTP load testing tool built out of a need to drill HTTP services with a constant request rate. It can be used both as a

Jan 7, 2023
A WebDriver client and acceptance testing library for Go
A WebDriver client and acceptance testing library for Go

Agouti Agouti is a library for writing browser-based acceptance tests in Google Go. It provides Gomega matchers and plays nicely with Ginkgo or Spec.

Dec 26, 2022
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! 🍕
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! 🍕

testza ?? Testza is like pizza for Go - you could life without it, but why should you? Get The Module | Documentation | Contributing | Code of Conduct

Dec 10, 2022