Randomized testing for Go

go-fuzz: randomized testing for Go

Go-fuzz is a coverage-guided fuzzing solution for testing of Go packages. Fuzzing is mainly applicable to packages that parse complex inputs (both text and binary), and is especially useful for hardening of systems that parse inputs from potentially malicious users (e.g. anything accepted over a network).

Note: go-fuzz has recently added preliminary support for fuzzing Go Modules. See the section below for more details. If you encounter a problem with modules, please file an issue with details. A workaround might be to disable modules via export GO111MODULE=off.

Usage

First, you need to write a test function of the form:

func Fuzz(data []byte) int

Data is a random input generated by go-fuzz, note that in most cases it is invalid. The function must return 1 if the fuzzer should increase priority of the given input during subsequent fuzzing (for example, the input is lexically correct and was parsed successfully); -1 if the input must not be added to corpus even if gives new coverage; and 0 otherwise; other values are reserved for future use.

The Fuzz function must be in a package that go-fuzz can import. This means the code you want to test can't be in package main. Fuzzing internal packages is supported, however.

In its basic form the Fuzz function just parses the input, and go-fuzz ensures that it does not panic, crash the program, allocate insane amount of memory nor hang. Fuzz function can also do application-level checks, which will make testing more efficient (discover more bugs). For example, Fuzz function can serialize all inputs that were successfully deserialized, thus ensuring that serialization can handle everything deserialization can produce. Or, Fuzz function can deserialize-serialize-deserialize-serialize and check that results of first and second serialization are equal. Or, Fuzz function can feed the input into two different implementations (e.g. dumb and optimized) and check that the output is equal. To communicate application-level bugs Fuzz function should panic (os.Exit(1) will work too, but panic message contains more info). Note that Fuzz function should not output to stdout/stderr, it will slow down fuzzing and nobody will see the output anyway. The exception is printing info about a bug just before panicking.

Here is an example of a simple Fuzz function for image/png package:

package png

import (
	"bytes"
	"image/png"
)

func Fuzz(data []byte) int {
	png.Decode(bytes.NewReader(data))
	return 0
}

A more useful Fuzz function would look like:

func Fuzz(data []byte) int {
	img, err := png.Decode(bytes.NewReader(data))
	if err != nil {
		if img != nil {
			panic("img != nil on error")
		}
		return 0
	}
	var w bytes.Buffer
	err = png.Encode(&w, img)
	if err != nil {
		panic(err)
	}
	return 1
}

The second step is collection of initial input corpus. Ideally, files in the corpus are as small as possible and as diverse as possible. You can use inputs used by unit tests and/or generate them. For example, for an image decoding package you can encode several small bitmaps (black, random noise, white with few non-white pixels) with different levels of compressions and use that as the initial corpus. Go-fuzz will deduplicate and minimize the inputs. So throwing in a thousand of inputs is fine, diversity is more important.

Put the initial corpus into the workdir/corpus directory (in our case examples/png/corpus). Go-fuzz will add own inputs to the corpus directory. Consider committing the generated inputs to your source control system, this will allow you to restart go-fuzz without losing previous work.

The go-fuzz-corpus repository contains a bunch of examples of test functions and initial input corpuses for various packages.

The next step is to get go-fuzz:

$ go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build

Then, download the corpus and build the test program with necessary instrumentation:

$ go get -d github.com/dvyukov/go-fuzz-corpus
$ cd $GOPATH/src/github.com/dvyukov/go-fuzz-corpus
$ cd png
$ go-fuzz-build

This will produce png-fuzz.zip archive.

Now we are ready to go:

$ go-fuzz

Go-fuzz will generate and test various inputs in an infinite loop. Workdir is used to store persistent data like current corpus and crashers, it allows fuzzer to continue after restart. Discovered bad inputs are stored in workdir/crashers dir; where file without a suffix contains binary input, file with .quoted suffix contains quoted input that can be directly copied into a reproducer program or a test, file with .output suffix contains output of the test on this input. Every few seconds go-fuzz prints logs to stderr of the form:

2015/04/25 12:39:53 workers: 500, corpus: 186 (42s ago), crashers: 3,
     restarts: 1/8027, execs: 12009519 (121224/sec), cover: 2746, uptime: 1m39s

Where workers means number of tests running in parallel (set with -procs flag). corpus is current number of interesting inputs the fuzzer has discovered, time in brackets says when the last interesting input was discovered. crashers is number of discovered bugs (check out workdir/crashers dir). restarts is the rate with which the fuzzer restarts test processes. The rate should be close to 1/10000 (which is the planned restart rate); if it is considerably higher than 1/10000, consider fixing already discovered bugs which lead to frequent restarts. execs is total number of test executions, and the number in brackets is the average speed of test executions. cover is number of bits set in a hashed coverage bitmap, if this number grows fuzzer uncovers new lines of code; size of the bitmap is 64K; ideally cover value should be less than ~5000, otherwise fuzzer can miss new interesting inputs due to hash collisions. And finally uptime is uptime of the process. This same information is also served via http (see the -http flag).

Modules support

go-fuzz has preliminary support for fuzzing Go Modules. go-fuzz respects the standard GO111MODULE environment variable, which can be set to on, off, or auto.

go-fuzz-build will add a require for github.com/dvyukov/go-fuzz to your go.mod. If desired, you may remove this once the build is complete.

Vendoring with modules is not yet supported. A vendor directory will be ignored, and go-fuzz will report an error if GOFLAGS=-mod=vendor is set.

Note that while modules are used to prepare the build, the final instrumented build is still done in GOPATH mode. For most modules, this should not matter.

libFuzzer support

go-fuzz-build can also generate an archive file that can be used with libFuzzer instead of go-fuzz (requires linux).

Sample usage:

$ cd $GOPATH/src/github.com/dvyukov/go-fuzz-corpus/fmt
$ go-fuzz-build -libfuzzer  # produces fmt.a
$ clang -fsanitize=fuzzer fmt.a -o fmt.libfuzzer
$ ./fmt.libfuzzer

When run with -libfuzzer, go-fuzz-build adds the additional build tag gofuzz_libfuzzer when building code.

Continuous Fuzzing

Just as unit-testing, fuzzing is better done continuously.

Currently there are 2 services that offer continuous fuzzing based on go-fuzz:

Random Notes

go-fuzz-build builds the program with gofuzz build tag, this allows to put the Fuzz function implementation directly into the tested package, but exclude it from normal builds with // +build gofuzz directive.

If your inputs contain a checksum, it can make sense to append/update the checksum in the Fuzz function. The chances that go-fuzz will generate the correct checksum are very low, so most work will be in vain otherwise.

Go-fuzz can utilize several machines. To do this, start the coordinator process separately:

$ go-fuzz -workdir=examples/png -coordinator=127.0.0.1:8745

It will manage persistent corpus and crashers and coordinate work of worker processes. Then run one or more worker processes as:

$ go-fuzz -bin=./png-fuzz.zip -worker=127.0.0.1:8745 -procs=10

External Articles

History rewrite

go-fuzz repository history was recently rewritten to exclude examples directory to reduce total repository size and download time (see #88, #114 and https://github.com/dvyukov/go-fuzz-corpus). Unfortunately, that means that go get -u command will fail if you had a previous version installed. Please remove $GOPATH/github.com/dvyukov/go-fuzz before running go get again.

Credits and technical details

Go-fuzz fuzzing logic is heavily based on american fuzzy lop, so refer to AFL readme if you are interested in technical details. AFL is written and maintained by Michal Zalewski. Some of the mutations employed by go-fuzz are inspired by work done by Mateusz Jurczyk, Gynvael Coldwind and Felix Gröbert.

Trophies

If you find some bugs with go-fuzz and are comfortable with sharing them, I would like to add them to this list. Please either send a pull request for README.md (preferable) or file an issue. If the source code is closed, you can say just "found N bugs in project X". Thank you.

Owner
Dmitry Vyukov
Dmitry Vyukov
https://github.com/dvyukov/go-fuzz
Comments
  • go-fuzz-build fails with Go 1.15 due to uncertain position of comments

    go-fuzz-build fails with Go 1.15 due to uncertain position of comments

    failed case 1:

    # yyc @ Yichens-MacBook-Pro in ~/GolandProjects/fuzz-test/fuzz on git:go-ast-bug-demo x [9:58:01] C:1
$ go-fuzz-build
failed to execute go build: exit status 2
# fuzz-test/fuzz
/Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:3: misplaced compiler directive

# yyc @ Yichens-MacBook-Pro in ~/GolandProjects/fuzz-test/fuzz on git:go-ast-bug-demo x [10:07:06] C:1
$ cat fuzzer.go      
package fuzz

//go:noescape
func foo() {}

func Fuzz(input []byte) int {
        foo()
        return 0
}

    go-fuzz-build will transform the input file to the following, which is obviously incorrect:

    //line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:1
package fuzz

//line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:1
import

//go:noescape
//line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:1
_go_fuzz_dep_ "go-fuzz-dep"

//line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:4
func foo()	{ _go_fuzz_dep_.CoverTab[22588]++ }

func Fuzz(input []byte) int {
//line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:6
	_go_fuzz_dep_.CoverTab[44810]++
								foo()
								return 0
}

//line /Users/yyc/GolandProjects/fuzz-test/fuzz/fuzzer.go:9
var _ = _go_fuzz_dep_.CoverTab

    I found this probably a go bug and submitted an issue: https://github.com/golang/go/issues/40546 I've also fixed this import issue with an ugly patch: https://github.com/oraluben/go-fuzz/tree/fix-import, but that does not overcome this issue, see case 2:

    failed case 2

    # yyc @ Yichens-MacBook-Pro in ~/GolandProjects/fuzz-test/fuzz on git:go-ast-bug-demo x [10:17:46] C:130
$ go-fuzz-build                             
failed to execute go build: exit status 2
# reflect
/Users/yyc/go/go1.15beta1/src/reflect/value.go:1425: misplaced compiler directive

# yyc @ Yichens-MacBook-Pro in ~/GolandProjects/fuzz-test/fuzz on git:go-ast-bug-demo x [10:17:57] C:1
$ cat fuzzer.go                                                               
package fuzz

import "reflect"

func Fuzz(input []byte) int {
        reflect.DeepEqual(1, 1)
        return 0
}

    the source: https://github.com/golang/go/blob/master/src/reflect/value.go#L1421-L1451 the transformed code:

    //line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1420
				return __gofuzz_v1 !=

//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1425
				//go:nocheckptr
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1420
				__gofuzz_v2
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1420
			}() == true
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1420
		default:
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1420
			_go_fuzz_dep_.CoverTab[7638]++
		}
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1421
	}
//line /Users/yyc/go/go1.15beta1/src/reflect/value.go:1421
	_go_fuzz_dep_.CoverTab[54290]++
								panic(&ValueError{"reflect.Value.OverflowUint", v.kind()})
}

    you can find //go:nocheckptr was inserted in the middle. I found it not easy to fix this without fixing the printer's logic, maybe you would have more idea about how to workaround this in go-fuzz-build?

  • preliminary Go modules support

    preliminary Go modules support

    Summary

    The heart of the change is very small -- we stop overwriting the user's value for the GO111MODULE env var in two places. This means we end up respecting whatever the user has set for GO111MODULE, and rely on cmd/go to interpret the meaning of an unset GO111MODULE (which defaults to auto for Go 1.11-1.13, and likely will default to on at some point during Go 1.14 development).

    In addition, there are some comments added, as well as an explicit check for GOFLAGS=-mod=vendor, which is not supported.

    The tests exercise 'replace' directives, v2+ modules, GO111MODULE set to on|off|auto, running inside GOPATH, running outside GOPATH, specifying pkg/func arguments to go-fuzz-build/go-fuzz, not specifying those arguments, the mechanisms by which go-fuzz/go-fuzz-dep is found, as well as vendoring (which is not supported, but it tests that a sensible error is emitted).

    Fixes #195

    Additional Details

    Historically, go-fuzz has set up a clean GOPATH environment in a temp directory, instrumented the code for fuzzing coverage, and built the instrumented code in the temp directory.

    The current approach is to keep doing that (and not set up a full-blown modules-based environment for building), but use modules to find the code to instrument.

    v2+ modules are potentially problematic due to Semantic Import Versioning (where the major version is placed in the import path, such as github.com/my/mod/v2). However, it turns out to not be too bad to handle it. For a v2+ module created following the 'major branch' approach, the /vN is materialized into a physical directory. For example, the github.com/russross/blackfriday/v2 module ends up in the /tmp/blah/gopath/src/github.com/russross/blackfriday/v2 directory. This should be a valid layout, and people use this layout today (which is called the 'major subdirectory' approach for a v2+ module).

    The approach does not copy go.mod files at all. (Before starting, I was thinking I would need to copy go.mod files to trigger "minimal module awareness", which is a transitional mechanism within cmd/go, but after some experimentation / thinking, I concluded it was not needed based on materializing the /vN).

    For tests, I am using the script-driven test facility that the core Go team created for testing cmd/go, which is well tuned to testing the various permutations of modules, including it lets you relatively easily set up go.mod files and source files, run commands and check results. (Some more details of the capability here: https://github.com/golang/go/blob/master/src/cmd/go/testdata/script/README).

  • make go-fuzz-build work with modules (for #195)

    make go-fuzz-build work with modules (for #195)

    Empirically, this change makes go-fuzz-build work with modules.

    For testing I'm using https://github.com/gomarkdown/markdown package which uses go modules (i.e. has go.mod file).

    Testing steps:

    • just to make sure, rm -rf ${GOPATH}/src/github.com/gomarkdown/markdown. Things might accidentally work if this is present
    • git clone https://github.com/gomarkdown/markdown.git to a directory that is outside of GOPATH e.g. ~/src/markdown
    • cd ~/src/markdown

    Before this change we get the following error:

    markdown kjk$ go-fuzz-build -x .
markdown.go:7:2: cannot find package "github.com/gomarkdown/markdown/ast" in any of:
	/usr/local/Cellar/go/1.12.9/libexec/src/github.com/gomarkdown/markdown/ast (from $GOROOT)
	/Users/kjk/go/src/github.com/gomarkdown/markdown/ast (from $GOPATH)

    The error comes from c.loadPkg(pkg) // load and typecheck pkg step, although all the heavy lifting is done by /x/tools library.

    This is not surprising. go-fuzz-build forces GO111MODULE=off which means that the tooling doesn't understand go.mod file. Since github.com/gomarkdown/markdown uses github.com/gomarkdown/markdown/ast, the tooling is trying to find that code in GOPATH but it's not there, so it fails.

    After this change, which merely sets GO111MODULE=auto, both go-fuzz-build -x github.com/gomarkdown/markdown and go-fuzz-build -x . complete the build.

    This is not unexpected - we tell the tools to figure out modules vs. GOPATH and act accordingly.

    I've also had a version that checked if go.mod file exists and setting GO111MODULE=off or GO111MODULE=on based on that. It also worked.

    I can't say this will work on every module-enabled repo but it does fix the problem on the one I've tested.

    cc @yevgenypats

  • go-fuzz-build doesn't work with go modules (go 1.11.2)

    go-fuzz-build doesn't work with go modules (go 1.11.2)

    FTR: I'm going to investigate this issue a bit. If the problem is on my side, I will close the issue.

    Environment:

    > go version
go version go1.11.2 darwin/amd64

> go env GOROOT GOPATH
/usr/local/Cellar/go/1.11.2/libexec
/Users/asd/go

    Steps to reproduce inside GOPATH:

    > pwd
/Users/asd/go/src/github.com/sashka/signedvalue

> go-fuzz-build github.com/sashka/signedvalue
failed to execute go build: exit status 1
go: finding github.com/dvyukov/go-fuzz v0.0.0-20181106053552-383a81f6d048
go: finding github.com/stephens2424/writerset v0.0.0-20150719204953-fe01f9c9e73f
go: finding github.com/elazarl/go-bindata-assetfs v1.0.0
go: finding github.com/sashka/signedvalue/go.fuzz.main latest
can't load package: package github.com/sashka/signedvalue/go.fuzz.main: unknown import path "github.com/sashka/signedvalue/go.fuzz.main": cannot find module providing package github.com/sashka/signedvalue/go.fuzz.main

    Steps to reproduce outside GOPATH:

    > cd /tmp
> git clone [email protected]:sashka/signedvalue
Cloning into 'signedvalue'...
...
> cd signedvalue/
> go-fuzz-build github.com/sashka/signedvalue
package dir 'go: finding github.com/dvyukov/go-fuzz/go-fuzz-dep latest
go: finding github.com/dvyukov/go-fuzz latest
/Users/asd/go/pkg/mod/github.com/dvyukov/[email protected]/go-fuzz-dep' does not end with import path 'github.com/dvyukov/go-fuzz/go-fuzz-dep'
  • More breakage with go 1.6 rc1

    More breakage with go 1.6 rc1 

    <dgryski@kaepora[go-fastlz] \ʕ◔ϖ◔ʔ/ > go-fuzz-build github.com/dgryski/go-fastlz
failed to execute go build: exit status 1
can't load package: /tmp/go-fuzz-build640820997/src/go-fuzz-main/main.go:5:2: non-standard import "github.com/dgryski/go-fastlz" in standard package "go-fuzz-main"
/tmp/go-fuzz-build640820997/src/encoding/binary/binary.go:22:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "encoding/binary"
/tmp/go-fuzz-build640820997/src/io/io.go:13:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "io"
/tmp/go-fuzz-build640820997/src/math/abs.go:5:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "math"
/tmp/go-fuzz-build640820997/src/reflect/deepequal.go:7:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "reflect"
/tmp/go-fuzz-build640820997/src/strconv/atob.go:5:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "strconv"
/tmp/go-fuzz-build640820997/src/unicode/utf8/utf8.go:7:8: non-standard import "github.com/dvyukov/go-fuzz/go-fuzz-dep" in standard package "unicode/utf8"

    Looks like this came from https://github.com/golang/go/commit/4223675913762a12cd23871fbd003d8a68cb49a1 From the commit message:

     Enforce that no standard package can depend on a non-standard one.
  • Compilation fails: Functions are undefined

    Compilation fails: Functions are undefined

    Hi!

    I'm not a golang pro, but I feel like I'm going crazy. I try to compile a fuzz case for a geth component. There are already some cases implemented in the code namely: https://github.com/ethereum/go-ethereum/blob/master/core/vm/runtime/fuzz.go. I wrote a different case with this code:

    // +build gofuzz

package p2p

func Fuzz(data []byte) int {
        closer, rw, p, _ := testPeer(nil)
        defer closer()
        if err := Senditems(rw, &data, data); err != nil {
                return 0;
        }
        return 1
}

    I placed the fuzz.go file in the p2p subdirectory. testpeer and SendItems are defined in the peer_test.go and message.go in the same directory (p2p) and package. While I can indeed compile the already added fuzz case with ~/go/bin/go-fuzz-build -tags nocgo github.com/ethereum/go-ethereum/core/vm/runtime I can't compile with

    ~/go/bin/go-fuzz-build -tags nocgo github.com/ethereum/go-ethereum/p2p
failed to execute go build: exit status 2
# github.com/ethereum/go-ethereum/p2p
p2p/Fuzz.go:6:22: undefined: testPeer
p2p/Fuzz.go:8:12: undefined: Senditems

    I really don't understand how this is not working... The fuzzcase inside the geth package doesn't seem to make anything different. Please help me understand.

  • go-fuzz does not work with cgo

    go-fuzz does not work with cgo

    Hello, I previously used go-fuzz with Go 1.4.2, but now that I'm on 1.5.1, it no longer seems to work. In addition, when switching back to 1.4.2, it doesn't seem to work anymore either.

    I just nuked my $GOPATH and reinstalled from scratch, just to see what would happen. It still doesn't seem to work. Any ideas?

    Thanks for your time.

    [zsh|matt@nerr-2]:~/git/go 0 % go version
go version go1.5.1 linux/amd64
[zsh|matt@nerr-2]:~/git/go 0 % go get github.com/mdlayher/ethernet                                                                                                                                       [zsh|matt@nerr-2]:~/git/go 2 % go get github.com/dvyukov/go-fuzz/go-fuzz                                                                            
[zsh|matt@nerr-2]:~/git/go 0 % go get github.com/dvyukov/go-fuzz/go-fuzz-build
[zsh|matt@nerr-2]:~/git/go 0 % cd src/github.com/mdlayher/ethernet 
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 (master) ± cat fuzz.go 
// +build gofuzz

package ethernet

func Fuzz(data []byte) int {
        f := new(Frame)
        if err := f.UnmarshalBinary(data); err != nil {
                return 0
        }

        if _, err := f.MarshalBinary(); err != nil {
                panic(err)
        }

        if err := f.UnmarshalFCS(data); err != nil {
                return 0
        }

        if _, err := f.MarshalFCS(); err != nil {
                panic(err)
        }

        return 1
}
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 (master) ± go-fuzz-build github.com/mdlayher/ethernet
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 *(master) ± go-fuzz -bin=./ethernet-fuzz.zip -workdir=fuzz/
2015/09/16 11:19:51 slaves: 8, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2015/09/16 11:19:54 slaves: 8, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s
2015/09/16 11:19:57 slaves: 8, corpus: 1 (9s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 9s
2015/09/16 11:20:00 slaves: 8, corpus: 1 (12s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 12s
2015/09/16 11:20:03 slaves: 8, corpus: 1 (15s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 15s
2015/09/16 11:20:06 slaves: 8, corpus: 1 (18s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 18s
2015/09/16 11:20:09 slaves: 8, corpus: 1 (21s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 21s
2015/09/16 11:20:12 slaves: 8, corpus: 1 (24s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 24s
2015/09/16 11:20:15 slaves: 8, corpus: 1 (27s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 27s
^C2015/09/16 11:20:17 shutting down...
  • Q: Debugging abysmal execution numbers

    Q: Debugging abysmal execution numbers

    Ahoi!

    I'm trying to toy with go-fuzz by fuzzing the MCL language lexer/parser of mgmt. Unfortunately my tests are extremely slow on both macos and linux when go-fuzz calls my Fuzz function. That is not so when calling the function myself.

    My overly simple fuzz target in fuzz/fuzz.go looks like this:

    package fuzz

import (
	"bytes"
	"github.com/purpleidea/mgmt/lang"
)

func Fuzz(data []byte) int {
	lang.LexParse(bytes.NewReader(data))
	return 0
}

    fuzz/corpus/third.mcl only contains:

    $three = 3

    I'm utilizing this target via a simple main in main-fuzz.go like this:

    package main

import (
	"io/ioutil"
	"github.com/purpleidea/mgmt/fuzz"
	"net/http"
	_ "net/http/pprof"
)

func main() {
	// we need a webserver to get the pprof webserver
	go func() {
		http.ListenAndServe("localhost:6060", nil)
	}()

	dat, _ := ioutil.ReadFile("./fuzz/corpus/third.mcl")
	for i := 1; i <= 1000; i++ {
		fuzz.Fuzz(dat)
	}
}

    Running this takes only a few seconds. Not amazingly fast, but not too shabby either:

    space@FeTAp ..epos/go/src/github.com/purpleidea/mgmt (git)-[master] % time go run main-fuzz.go
go run main-fuzz.go  2.44s user 1.02s system 106% cpu 3.251 total

    Unfortunately, when building and running it with go-fuzz, I get abysmal execution numbers:

    space@FeTAp ..go/src/github.com/purpleidea/mgmt/fuzz (git)-[master] % go-fuzz --procs=1 -bin=./fuzz-fuzz.zip -workdir=.
2020/02/18 12:52:26 workers: 1, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2020/02/18 12:52:29 workers: 1, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s
2020/02/18 12:52:32 workers: 1, corpus: 1 (9s ago), crashers: 0, restarts: 1/22, execs: 22 (2/sec), cover: 0, uptime: 9s
2020/02/18 12:52:35 workers: 1, corpus: 1 (12s ago), crashers: 0, restarts: 1/35, execs: 35 (3/sec), cover: 0, uptime: 12s
2020/02/18 12:52:38 workers: 1, corpus: 1 (15s ago), crashers: 0, restarts: 1/37, execs: 37 (2/sec), cover: 0, uptime: 15s
2020/02/18 12:52:41 workers: 1, corpus: 1 (18s ago), crashers: 0, restarts: 1/42, execs: 42 (2/sec), cover: 0, uptime: 18s
2020/02/18 12:52:44 workers: 1, corpus: 1 (21s ago), crashers: 0, restarts: 1/44, execs: 44 (2/sec), cover: 0, uptime: 21s
2020/02/18 12:52:47 workers: 1, corpus: 1 (24s ago), crashers: 0, restarts: 1/44, execs: 44 (2/sec), cover: 0, uptime: 24s
2020/02/18 12:52:50 workers: 1, corpus: 1 (27s ago), crashers: 0, restarts: 1/49, execs: 49 (2/sec), cover: 0, uptime: 27s
2020/02/18 12:52:53 workers: 1, corpus: 1 (30s ago), crashers: 0, restarts: 1/51, execs: 51 (2/sec), cover: 0, uptime: 30s

    Any tips how to how to debug this. Can I utilize pprof from inside go-fuzz somehow?

  • go-fuzz-build with '.'

    go-fuzz-build with '.' 

    andrew@andrew-laptop:fuzz $ go-fuzz-build .
failed to execute go build: exit status 1
can't load package: /tmp/go-fuzz-build703235414/src/go-fuzz-main/main.go:5:2: local import "." in non-local package
import cycle not allowed
package go-fuzz-main
    imports .
    imports .

    I don't mind if you don't care to support this.

  • Fuzzing doesn't work (no cgo)

    Fuzzing doesn't work (no cgo)

    Hello, I have a fuzzing job that doesn't run when it should. At first it was because of CGO but now I have CGO disabled and it still doesn't works. I'm not sure how to debug this.

    Env

    $ go env GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOOS="darwin" GOPATH="/Users/joss/projects/go" GORACE="" GOROOT="/usr/local/Cellar/go/1.9.2/libexec" GOTOOLDIR="/usr/local/Cellar/go/1.9.2/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" CC="clang" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3g/vj270kpd74374rcl0yv9c5dm0000gn/T/go-build339987089=/tmp/go-build -gno-record-gcc-switches -fno-common" CXX="clang++" CGO_ENABLED="1" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config"

    Description

    Fuzzing this code fails:

    package fuzz

import (
	_ "github.com/ethereum/go-ethereum/core"
)

func Fuzz(data []byte) int {
	return 0
}

    I'm fuzzing using go-ethereum/core version bfdc0fa3622d7c3b421d2f5a6dda5746be41bfde I've stubbed the crypto/secp256k1 package as follow: https://gist.github.com/josselin-c/9c66bac044e1489c290c37c43250ff28

    I'm building the fuzzing code with cgo disabled: `CGO_ENABLED=0 go-fuzz-build github.com/ethereum/go-ethereum/tests/vmfuzz``

    The output:

    go-fuzz -bin=./fuzz-fuzz.zip -workdir=results
2017/11/09 08:46:09 slaves: 4, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2017/11/09 08:46:12 slaves: 4, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s
2017/11/09 08:46:15 slaves: 4, corpus: 1 (9s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 9s
2017/11/09 08:46:18 slaves: 4, corpus: 1 (12s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 12s
  • added a basic web frontend

    added a basic web frontend

    I wanted an easy way to check up on the fuzzer running from my server, so I wrote up this little frontend. I kept it pretty simple for the time being. Let me know if you'd like me to change anything to merge, but I thought you might like this as part of the project.

  • go-fuzz-dep: sonar serialization contains faulty type assumption

    go-fuzz-dep: sonar serialization contains faulty type assumption

    While fuzzing something, I got this crash:

    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x10210d414]

goroutine 1 [running]:
go-fuzz-dep.serialize({0x0, 0x0}, {0x102205300?, 0x140000aa900?}, {0x1400011db60?, 0xbdbfefbfbdef2f20?, 0x1400011db78?})
	go-fuzz-dep/sonar.go:150 +0xd94
go-fuzz-dep.Sonar({0x0, 0x0}, {0x102205300, 0x140000aa900}, 0x7c700)
	go-fuzz-dep/sonar.go:31 +0x54
reflect.DeepEqual.func2(...)
	/Users/josh/go/1.18/src/reflect/deepequal.go:230
reflect.DeepEqual({0x0, 0x0}, {0x102205300?, 0x140000aa900})
	/Users/josh/go/1.18/src/reflect/deepequal.go:230 +0xcc
[...]

    The relevant reflect.DeepEqual code is:

    func DeepEqual(x, y any) bool {
	if x == nil || y == nil {
		return x == y  // <------
	}

    The relevant sonar.go line is:

    func serialize(v, v2 interface{}, buf []byte) (n, flags uint8) {
	switch vv := v.(type) {
        // omit many cases
	default:
		// Special case: string literal is compared with a variable of
		// user type with string underlying type:
		//	type Name string
		//	var name Name
		//	if name == "foo" { ... }
		if _, ok := v2.(string); ok {
			s := *(*string)((*iface)(unsafe.Pointer(&v)).val)  // <--------
			if len(s) <= SonarMaxLen {
				return uint8(copy(buf[:], s)), SonarString
			}
		}
        // ...

    I think this code contains a faulty assumption. If v has an interface type (as with reflect.DeepEqual), it can be compared with a string without itself being a string. The iface conversion is thus unsafe. I believe in this case it is due to a nil interface being compared with a string, causing a nil pointer dereference when reading the iface.val field.

    If anyone cares (unlikely), this should be made safer, probably by using reflect.Type.

  • go 1.18 version corpus is not increase on Windows

    go 1.18 version corpus is not increase on Windows

    go 1.18 version

    why this version i build gofuzz corups is not increase ah I don not know where wrong

    D:\code\gofuzz\src\mail> go-fuzz 2022/06/07 20:21:17 workers: 8, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s 2022/06/07 20:21:20 workers: 8, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s 2022/06/07 20:21:23 workers: 8, corpus: 1 (9s ago), crashers: 0, restarts: 1/1, execs: 185 (21/sec), cover: 0, uptime: 9s 2022/06/07 20:21:26 workers: 8, corpus: 1 (12s ago), crashers: 0, restarts: 1/1, execs: 370 (31/sec), cover: 0, uptime: 12s2022/06/07 20:21:29 workers: 8, corpus: 1 (15s ago), crashers: 0, restarts: 1/1, execs: 555 (37/sec), cover: 0, uptime: 15s2022/06/07 20:21:32 workers: 8, corpus: 1 (18s ago), crashers: 0, restarts: 1/1, execs: 739 (41/sec), cover: 0, uptime: 18s2022/06/07 20:21:35 workers: 8, corpus: 1 (21s ago), crashers: 0, restarts: 1/1, execs: 924 (44/sec), cover: 0, uptime: 21s2022/06/07 20:21:38 workers: 8, corpus: 1 (24s ago), crashers: 0, restarts: 1/1, execs: 1107 (46/sec), cover: 0, uptime: 24

    my project

    image

  • Add new mutation strategies

    Add new mutation strategies

    Hey,

    We have had an internship project in Trail of Bits to improve go-fuzz recently which was done by @vfsrfs.

    We are aware of the ongoing official work on native fuzzing support but since we still rely on go-fuzz we went ahead to improve its pain points and so that's why we propose this PR. Feel free to drop it if you feel it is too much or you do not want to introduce any changes in go-fuzz.

    Below I am pasting the description from the original PR merged to our fork of go-fuzz (https://github.com/trailofbits/go-fuzz/pull/3).

    This PR adds new mutation strategies for the inputs:

    • InsertRepeatedBytes: A sequence of the same bytes is inserted at a random position into the input
    • ShuffleBytes: The bytes of a subslice of the input are shuffled to random positions
    • LEB128 Encoding: Integer Literals can be encoded with (unsigned | signed) Little Endian Base 128
  • Add support for token dictionaries

    Add support for token dictionaries

    Hey,

    We have had an internship project in Trail of Bits to improve go-fuzz recently which was done by @vfsrfs.

    We are aware of the ongoing official work on native fuzzing support but since we still rely on go-fuzz we went ahead to improve its pain points and so that's why we propose this PR. Feel free to drop it if you feel it is too much or you do not want to introduce any changes in go-fuzz.

    Below I am pasting the description from the original PR merged to our fork of go-fuzz (https://github.com/trailofbits/go-fuzz/pull/2).

    This PR adds support for dictionaries containing interesting keywords (tokens) that are useful for the mutation of inputs while fuzzing, particularly, when fuzzing syntax-aware programs (#174). This modification allows to provide the -dict flag to go-fuzz, so that the user can provide a dictionary file with useful tokens for the fuzzing campaign. E.g.:

    -dict /path/dictionary.dict

    The tokens parsed from the dictionary are stored in ROData.strLits, as those are the string literals that are used by the mutator engine when generating new fuzzing inputs.

    The dictionary format that is accepted by the -dict flag is the same that is used by AFL/Libfuzzer (see https://github.com/google/AFL/tree/master/dictionaries).

    This dictionary format defines that there is one token per line. Every line consists of a name followed by an equal sign and the token in quotes (e.g. name=”token”). It is also possible to define binary sequences by providing the values in hex (e.g. \xNN) within the token. To insert a backslash or a double quote within the token, it has to be escaped using a backslash (e.g. \\ or \”). \n and \t are recognized as well, since they might be useful for text-based protocols. Other problematic characters can be added by providing its hex value.

    To make this implementation fully compatible with AFL/Libfuzzer’s dictionaries, token levels are supported. A level can be appended to every token, by appending @<num> to the keyword, e.g. keyword@1=”token”

    These tokens will be loaded only, if the dictionary level is equal to or greater than the specified number. The default dictionary level is 0, but it can be increased by appending @<num> to the dictionary path. E.g.:

    -dict /path/dictionary.dict@1
  • Fix the

    Fix the "undefined: fs in fs.FileMode" issue

    Hey,

    We have had an internship project in Trail of Bits to improve go-fuzz recently which was done by @vfsrfs.

    We are aware of the ongoing official work on native fuzzing support but since we still rely on go-fuzz we went ahead to improve its pain points and so that's why we propose this PR. Feel free to drop it if you feel it is too much or you do not want to introduce any changes in go-fuzz.

    Below I am pasting the description from a PR merged to our fork of go-fuzz (https://github.com/trailofbits/go-fuzz/pull/1).

    Issue

    This PR addresses the issue that is described in #325

    Reproduction

    When compiling the instrumented version of the code below, go-fuzz-build crashes with the error undefined: fs in fs.FileMode.

    package homedir

import (
	"os"
	"fmt"
)

func HomeDir() {
	p := "text.txt"
	info, _ := os.Stat(p)

	if info.Mode().Perm()&(1<<(uint(7))) != 0 {
		fmt.Println("crash")
	}
}

    Cause

    Looking into the instrumented code, we can see that the expression

    info.Mode().Perm()&(1<<(uint(7)))

    is instrumented to

    __gofuzz_v1 := fs.FileMode(info.Mode().Perm() & (1 << 7)).

    At the first glance we realize that the type conversion to fs.FileMode is causing this crash.

    The underlying issue is that the fs package is not imported and therefore it shouldn’t be referenced in the instrumented code. This bug first appeared in go1.16, since in that version a type alias in the os package was introduced that mapped os.FileMode to fs.FileMode (see https://go.dev/src/os/types.go?s=798:825#L18). So in fact, this type conversion should have been os.FileMode instead of fs.FileMode. However, when the code is loaded and parsed for instrumentation by /x/tool/packages, the type-checker already resolves the type alias, so that we only get the type that is included in fs instead of the one included in os.

    Fix

    To fix this issue, I propose to scan every type that is identified by the /x/tool/packages type-checker and to do a lookup of the package the where types are defined. If there is a type that is defined in a package which is not imported, the instrumentation will add the corresponding import statement. This ensures, that if there is an explicit type conversion using that type (which is the root cause of this bug), we will still be able to compile the instrumented code. The semantics of the instrumented code should still be the same after adding the import statement, because the package must have been imported by one of the imports of the analyzed code (e.g. os imports fs), so that the initialization of the package is still being made. Since there is no guarantee, that there will be an explicit type conversion using the newly imported package, we must make sure to remove the unused imports. For this purpose, we run goimports to clear every unused import and then proceed to compile the instrumented code.

  • Missing textflag.h

    Missing textflag.h

    I'm trying to install de but get an error:

    # go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build

# github.com/dvyukov/go-fuzz/go-fuzz
/root/go/pkg/mod/github.com/dvyukov/[email protected]/go-fuzz/compare_amd64.s:4:10: fatal error: textflag.h: No such file or directory
    4 | #include "textflag.h"
      |          ^~~~~~~~~~~~
compilation terminated.


    # uname                                                                                                                                                             130 ⨯
Linux
# go version                                                                                                                                                        130 ⨯
go version go1.16.5 gccgo (Debian 11.2.0-13) 11.2.0 linux/amd64

    Thanks,

Related tags
Testing Frameworks testing go fuzzing
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Nov 24, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.
 Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Jan 2, 2023
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Jan 6, 2023
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Dec 13, 2022
Simple Go snapshot testing
 Simple Go snapshot testing

Incredibly simple Go snapshot testing: cupaloy takes a snapshot of your test output and compares it to a snapshot committed alongside your tests. If t

Jan 5, 2023
Clean database for testing, inspired by database_cleaner for Ruby

DbCleaner Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood to make sure the test can runs in pa

Nov 17, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Sep 27, 2022
API testing framework inspired by frisby-js
 API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

Sep 27, 2022
Mutation testing for Go source code

go-mutesting go-mutesting is a framework for performing mutation testing on Go source code. Its main purpose is to find source code, which is not cove

Dec 28, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
 Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022
Minimal and Beautiful Go testing framework
 Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

Dec 25, 2022
A Comprehensive Coverage Testing System for The Go Programming Language
 A Comprehensive Coverage Testing System for The Go Programming Language

goc 中文页 | goc is a comprehensive coverage testing system for The Go Programming Language, especially for some complex scenarios, like system testing c

Jan 8, 2023
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
 Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

Dec 30, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Dec 16, 2022
Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

Oct 28, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

Jan 4, 2023
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Jan 5, 2023
A Go library help testing your RESTful API application

RESTit A Go micro-framework to help writing RESTful API integration test Package RESTit provides helps to those who want to write an integration test

Oct 28, 2022