Fuzz testing for go.

gofuzz

gofuzz is a library for populating go objects with random values.

GoDoc Travis

This is useful for testing:

  • Do your project's objects really serialize/unserialize correctly in all cases?
  • Is there an incorrectly formatted object that will cause your project to panic?

Import with import "github.com/google/gofuzz"

You can use it on single variables:

f := fuzz.New()
var myInt int
f.Fuzz(&myInt) // myInt gets a random value.

You can use it on maps:

f := fuzz.New().NilChance(0).NumElements(1, 1)
var myMap map[ComplexKeyType]string
f.Fuzz(&myMap) // myMap will have exactly one element.

Customize the chance of getting a nil pointer:

f := fuzz.New().NilChance(.5)
var fancyStruct struct {
  A, B, C, D *string
}
f.Fuzz(&fancyStruct) // About half the pointers should be set.

You can even customize the randomization completely if needed:

type MyEnum string
const (
        A MyEnum = "A"
        B MyEnum = "B"
)
type MyInfo struct {
        Type MyEnum
        AInfo *string
        BInfo *string
}

f := fuzz.New().NilChance(0).Funcs(
        func(e *MyInfo, c fuzz.Continue) {
                switch c.Intn(2) {
                case 0:
                        e.Type = A
                        c.Fuzz(&e.AInfo)
                case 1:
                        e.Type = B
                        c.Fuzz(&e.BInfo)
                }
        },
)

var myObject MyInfo
f.Fuzz(&myObject) // Type will correspond to whether A or B info is set.

See more examples in example_test.go.

You can use this library for easier go-fuzzing. go-fuzz provides the user a byte-slice, which should be converted to different inputs for the tested function. This library can help convert the byte slice. Consider for example a fuzz test for a the function mypackage.MyFunc that takes an int arguments:

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func Fuzz(data []byte) int {
        var i int
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(i)
        return 0
}

Happy testing!

Owner
Google
Google ❤️ Open Source
Google
Comments
  • Add support to fuzz from go-fuzz input

    Add support to fuzz from go-fuzz input

    Add a helper function that enables using gofuzz (this project) with go-fuzz for continuose fuzzing. Essentially, it enables translating the fuzzing bytes from go-fuzz to any Go object using this library.

    This change will enable using this project with fuzzing websites such as fuzzit.dev or fuzzbuzz.io.

    The underlying implementation was an idea of @lavalamp, by which a random source is created that generates random numbers from the input bytes slice. In this way changes in the source result in logical changes in the random data, which enables the fuzzer to efficiently search the space of inputs.

    Fixes #33

  • Add a FuzzNoCustom() function

    Add a FuzzNoCustom() function

    This skips custom fuzz functions. The use case is for custom-fuzzing a subset of fields of a larger struct. The custom function can call this on the struct (to avoid a recursive call) and then patch-up fields it needs to handle specially.

    Better solution to #10.

    Fixes #10

  • Optimize randBool() and randString()

    Optimize randBool() and randString()

    I love this library, small but gets the job done! :)

    I have tinkered a bit with the functions to generate bools and strings and made them a bit faster. Hopefully it will save someone some clocks when running tests.

    The code speeds up bool generation 3-fold. I've ran some benchmarks on GCP VM's and my local MacBook:

    My MacBook Pro:

    BenchmarkRandBool-8 75312272 14.5 ns/op BenchmarkRandBoolNew-8 235223112 5.06 ns/op BenchmarkRandStringStock-8 1439635 841 ns/op BenchmarkRandStringBuilder-8 1541736 775 ns/op

    8 core GCP E2:

    BenchmarkRandBool-8 70003590 16.6 ns/op BenchmarkRandBoolNew-8 236062108 5.08 ns/op BenchmarkRandStringStock-8 1370116 874 ns/op BenchmarkRandStringBuilder-8 1484086 807 ns/op

    8 core GCP N2 (Intel Cascade Lake):

    BenchmarkRandBool-8 91276062 13.1 ns/op BenchmarkRandBoolNew-8 301435986 3.98 ns/op BenchmarkRandStringStock-8 1743852 688 ns/op BenchmarkRandStringBuilder-8 1883102 638 ns/op

    8 core GCP N1 (Intel Skylake):

    BenchmarkRandBool-8 71896552 16.5 ns/op BenchmarkRandBoolNew-8 239149131 5.03 ns/op BenchmarkRandStringStock-8 1375860 873 ns/op BenchmarkRandStringBuilder-8 1484378 808 ns/op

    randString() will be 5-7% due to less data being moved around (thanks to strings.Builder)

    randString() memory usage change:

    BenchmarkRandStringStock-8 1543179 772 ns/op 71 B/op 1 allocs/op BenchmarkRandStringBuilder-8 1639436 735 ns/op 48 B/op 1 allocs/op

  • Allow skipping certain struct fields while fuzzing (based on a specified prefix)

    Allow skipping certain struct fields while fuzzing (based on a specified prefix)

    Protobuf generated code have certain exported fields (XXX_...) which cannot be set as that corrupts the serialization. This change allows specifying certain prefix for fields which are ignored while fuzzing a struct.

  • Set the character generation mode to generate character that conform to Unicode encoding

    Set the character generation mode to generate character that conform to Unicode encoding

    You can use f.WithStringGenMode(mode) to set generation mode to generate different character, like ASCII, MultiByte, CommonCJK, Chinese, Japanese, Number, Lower-Case, Upper-Case , default is random mode(each rune will be one of above character).

  • Fix time.Time fuzzing to generate only valid zone offsets

    Fix time.Time fuzzing to generate only valid zone offsets

    As described in #14 comment: this patch fixes the default fuzzer for time.Time to limit it's output to time values that will serialise correctly.

    My understanding of the issue is that while the time.Unix docs specify that:

    It is valid to pass nsec outside the range [0, 999999999]

    Doing so seems to break time.MarshalBinary which will return an error about the timezone offset being invalid.

    Arguably that could be considered a bug in time.Time but I think it's reasonable to just accommodate and only generate nanoseconds in the sub-second range.

  • Support fuzzing unexported fields

    Support fuzzing unexported fields

    The current gofuzz only support fuzzing the exported fields (i.e. the fields that start with an uppercase letter), other fields are simply filtered out according to this line.

    However, a lot of struct also contains un-exported fields (i.e. fields start with lowercase letters) and sometimes people want to do fuzzing on those structs. In this pr I fixed this problem by adding allowUnexportedFields to the fuzzer, which allows user to decide whether they want to include unexported fields while fuzzing. It is set to false by default. But it can be turned on with AllowUnexportedFields function.

    Tests are also added.

  • Fix charRange to allow picking the last item

    Fix charRange to allow picking the last item

    Before this change, choose would never pick the last character of the set. This is not the expected behavior: if the range is a-z, we expect a,b,...,y,z to be picked in a uniform way.

  • Allow Continue.Fuzz to accept a reflect.Value

    Allow Continue.Fuzz to accept a reflect.Value

    This is a small optimization, so I completely understand if you would rather keep the existing interface and close this PR without merging. I made the change before I realized I could use reflect.Value.Addr() as the argument. I thought I would open this PR anyway, in case there is interest in the small optimization. I imagine in some larger test suites it might save a little time by avoiding the need to reflect again.

    Continue.Fuzz (and FuzzNoCustom) immediately convert the interface{} into a reflect.Value. In some cases the caller may already have a reflect.Value, so accepting that type makes it easier to write custom fuzz functions.

    One use case for this is ignoring a field on a struct. The SkipFieldsWithPattern option is a good choice when the field name should always be ignored. In other cases a field name may exist on multiple structs, and using a regex pattern would result in the field being skipped in all cases. To ignore a field by name on only a single struct a custom fuzz function can use reflect to iterate over the fields, and call c.Fuzz() on all fields except for the ignored field.

  • Update import statement to reference correct library name

    Update import statement to reference correct library name

    The example in the documentation references the go-fuzz library rather than gofuzz, which appears to be a mistake. This just tweaks that so that the example should hopefully work when copied.

  • Fix testing methodology; passes accumulate now.

    Fix testing methodology; passes accumulate now.

    go test -count=10000 . passes now.

    I spot checked by introducing errors and the new tests continue to catch bugs. Running 10k tests as above caught the one place I messed up the fail/pass logic.

  • Add AllowUnsupportedFields to ignore interface/etc fields and not panic

    Add AllowUnsupportedFields to ignore interface/etc fields and not panic

    Description

    This adds a new property to the fuzzer: AllowUnsupportedFields. If used, when the fuzzer sees an unsupported field (like an interface, func or a channel) it will ignore it instead of panic. This resolves #27

  • NewFromGoFuzz doesn't work well with go-fuzz

    NewFromGoFuzz doesn't work well with go-fuzz

    I've come across projects using go-fuzz with NewFromGoFuzz where fuzzing didn't catch obvious bugs. I've distilled these down into very simple, unstructured cases (which you'd probably not use gofuzz for) which demonstrate the problems. In the projects in questions, I've used JSON and protobuf unmarshalling of go-fuzz provided data into structs with better results. Not sure if there's anything obvious to do with this information except to document the downsides?

    It's probably easiest to describe the problem with examples:

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(i int) {
        if i == 1337 {
            panic("found correct i")
        }
    }
    
    func Fuzz(data []byte) int {
        var i int
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(i)
        return 0
    }
    

    The above (an extension of the README example) would have to run for a very long time to find a crasher. Go-fuzz has its sonar to guide it to new coverage gated by comparisons e.g. the following finds crashers almost immediately:

    // +build gofuzz
    
    package mypackage
    
    import "encoding/binary"
    
    func MyFunc(i int) {
        if i == 1337 {
            panic("found correct i")
        }
    }
    
    func Fuzz(data []byte) int {
        i := binary.BigEndian.Uint64(data)
        MyFunc(int(i))
        return 0
    }
    

    Decoding the data from go-fuzz as little-endian, hex encoded strings or ascii will also result in the crasher being found quickly as the number "1337" is used in the mutator in all of these forms.

    I found that applying this patch

    diff --git a/bytesource/bytesource.go b/bytesource/bytesource.go
    index 5bb3659..efd6a65 100644
    --- a/bytesource/bytesource.go
    +++ b/bytesource/bytesource.go
    @@ -40,7 +40,9 @@ func New(input []byte) *ByteSource {
                    fallback: rand.NewSource(0),
            }
            if len(input) > 0 {
    -               s.fallback = rand.NewSource(int64(s.consumeUint64()))
    +               seed := make([]byte, 8)
    +               copy(seed, input)
    +               s.fallback = rand.NewSource(int64(binary.BigEndian.Uint64(seed)))
            }
            return s
     }
    

    allowed go-fuzz to work a little bit better for this scenario. The improvement comes because the first 8 bytes of the go-fuzz input are not lost to the random number generator seed.

    The following demonstrates how inputs from go-fuzz with string literals are unrecognisable after passing through NewFromGoFuzz.Fuzz:

    package main
    
    import (
        "fmt"
    
        fuzz "github.com/google/gofuzz"
    )
    
    func main() {
        data := []byte("-----BEGIN RSA PRIVATE KEY-----")
        var str string
        fuzz.NewFromGoFuzz(data).Fuzz(&str)
        fmt.Printf("str = %s\n", str)
    }
    

    outputs

    str = Ȱ#ǻŐ蒉
    

    Here, randString will consume at least 8*(n+1) bytes of the go-fuzz input (n is the length chosen for the string) in such a way as to "slice and dice" the input leaving it with no recognisable fragments. This prevents the fuzzer finding crashers for the following:

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(str string) {
        if str == "1337" {
            panic("found correct")
        }
    }
    
    func Fuzz(data []byte) int {
        var str string
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(str)
        return 0
    }
    

    An equivalent fuzzer without gofuzz

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(str string) {
        if str == "1337" {
            panic("found correct string")
        }
    }
    
    func Fuzz(data []byte) int {
        MyFunc([]byte(str))
        return 0
    }
    

    immediately finds a crasher. It should also be noted that strings from gofuzz will be limited in length and well-formed utf-8 which may be at odds with the aims for fuzzing.

  • Add note to docs that private fields of structs cannot be fuzzed using this package

    Add note to docs that private fields of structs cannot be fuzzed using this package

    I tried to use this package (through go-fuzz) for the first time when I was fuzzing a struct that has only private fields. It took me a while to figure out that this package doesn't put any values into private fields...

    Now, I'm not terribly familiar with reflection in Go but from what I can tell based on the reflection docs and some testing is that it would be impossible for this package to set the value of private fields. Am i correct?

    If the above statement is correct, I think it would be useful if there was a note in the README (or somewhere else in the docs) stating that private fields cannot be populated by this package. Thoughts?

  • issue building with go 1.11.3 and previous

    issue building with go 1.11.3 and previous

    9532c33cb8812a3a439858750fb792cb88d35495 is causing versions prior to go 1.11.4 to have issues requiring the module.

    e.g. semaphore v1 has go 1.11.0 as latest version and builds fail with:

    go build github.com/google/gofuzz: module requires Go 1.12
    

    note: travis-ci hasn't tested the last 2 commits on master.

  • presence of an interface value always panics

    presence of an interface value always panics

    Hello there, thank you for this project!

    I am currently looking at using it to fuzz inputs to https://github.com/census-instrumentation/opencensus-service

    However, if a struct has a field with a value whose type is an interface, regardless of if that field is set or not. For example

    type A struct {
                Name string
                Debug bool
                Err error
    }
    a := new(A)
    f.Fuzz(a)
    

    gofuzz will always crash with

    On no value set

    panic: Can't handle <nil>: interface [recovered]
    	panic: Can't handle <nil>: interface
    

    On value set

    panic: Can't handle &errors.errorString{s:"Foo"}: interface [recovered]
    	panic: Can't handle &errors.errorString{s:"Foo"}: interface
    

    Perhaps we could skip trying to mutate interface values that we don't know about and let tests proceed normally, otherwise it becomes unproductive to debug this cryptic error.

    Thank you.

Fuzz Go defer/panic/recover

deferfuzz deferfuzz is a fuzzer for Go defer/panic/recover. Caveat: I wrote this in a couple hours, and I'm more of a compiler engineer than a fuzzer

Mar 3, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Dec 12, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Nov 24, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Jan 2, 2023
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Jan 6, 2023
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Dec 13, 2022
Simple Go snapshot testing
Simple Go snapshot testing

Incredibly simple Go snapshot testing: cupaloy takes a snapshot of your test output and compares it to a snapshot committed alongside your tests. If t

Jan 5, 2023
Clean database for testing, inspired by database_cleaner for Ruby

DbCleaner Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood to make sure the test can runs in pa

Nov 17, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Sep 27, 2022
API testing framework inspired by frisby-js
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

Sep 27, 2022
Mutation testing for Go source code

go-mutesting go-mutesting is a framework for performing mutation testing on Go source code. Its main purpose is to find source code, which is not cove

Dec 28, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022
Minimal and Beautiful Go testing framework
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

Dec 25, 2022
A Comprehensive Coverage Testing System for The Go Programming Language
A Comprehensive Coverage Testing System for The Go Programming Language

goc 中文页 | goc is a comprehensive coverage testing system for The Go Programming Language, especially for some complex scenarios, like system testing c

Jan 8, 2023
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

Dec 30, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Dec 16, 2022
Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

Oct 28, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

Jan 4, 2023