Goroutine Leak Detector

Leaktest Build Status codecov Sourcegraph Documentation

Refactored, tested variant of the goroutine leak detector found in both net/http tests and the cockroachdb source tree.

Takes a snapshot of running goroutines at the start of a test, and at the end - compares the two and voila. Ignores runtime/sys goroutines. Doesn't play nice with t.Parallel() right now, but there are plans to do so.

Installation

Go 1.7+

go get -u github.com/fortytw2/leaktest

Go 1.5/1.6 need to use the tag v1.0.0, as newer versions depend on context.Context.

Example

These tests fail, because they leak a goroutine

// Default "Check" will poll for 5 seconds to check that all
// goroutines are cleaned up
func TestPool(t *testing.T) {
    defer leaktest.Check(t)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

// Helper function to timeout after X duration
func TestPoolTimeout(t *testing.T) {
    defer leaktest.CheckTimeout(t, time.Second)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

// Use Go 1.7+ context.Context for cancellation
func TestPoolContext(t *testing.T) {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    defer leaktest.CheckContext(ctx, t)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

LICENSE

Same BSD-style as Go, see LICENSE

Owner
Ian
building good things
Ian
Comments
  • Does not work with go 1.7

    Does not work with go 1.7

    With Go 1.7, there is always a leaked Gorouting reported, even for empty tests:

    func TestStoppedSlotReporter(t *testing.T) { defer leaktest.Check(t)() }

  • Timeouts happen immediately

    Timeouts happen immediately

    It appears that since the context.WithTimeout is called at the beginning of the test, tests which take longer than timeout to complete automatically fail. This is because the Done channel on the context is already closed by time the test completes. This is problematic for tests which take an non-deterministic amount of time to complete.

    I would have expected the timeout value to be the amount of time leaktest waits when the deferred function is executed, not from the moment Check is called.

    Currently we are getting around the issue by wrapping leaktest functionality:

    func CheckTimeout(t *testing.T, dur time.Duration) func() {
        ctx, cancel := context.WithCancel(context.Background())
        fn := leaktest.CheckContext(ctx, t)
        return func() {
            time.AfterFunc(dur, cancel)
    	fn()
        }
    }
    
  • Tests hang in simple for loop

    Tests hang in simple for loop

    Hello! I've create a minial step to create a condition of a leaked goroutine. In this case, the test never finishes.

    I think I've narrowed the bug down to this line where it seems to block.

    My guess is, because this goroutine never blocks and runtime.Stack() is waiting for the goroutine to block?

    package hub
    
    import (
    	"testing"
    	"time"
    
    	"github.com/fortytw2/leaktest"
    )
    
    func TestInfiniteLoop(t *testing.T) {
    	defer leaktest.CheckTimeout(t, time.Second)()
    	go func() {
    		i := 0
    		for {
    			i++
    		}
    	}()
    }
    

    System details

    go version go1.9 linux/amd64
    GOARCH="amd64"
    GOBIN=""
    GOEXE=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOOS="linux"
    GOPATH="/home/anthony/go"
    GORACE=""
    GOROOT="/usr/local/go"
    GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    CC="gcc"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build343347876=/tmp/go-build -gno-record-gcc-switches"
    CXX="g++"
    CGO_ENABLED="1"
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOROOT/bin/go version: go version go1.9 linux/amd64
    GOROOT/bin/go tool compile -V: compile version go1.9
    uname -sr: Linux 4.10.0-40-generic
    Distributor ID:	Ubuntu
    Description:	Ubuntu 16.04.3 LTS
    Release:	16.04
    Codename:	xenial
    /lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.23-0ubuntu9) stable release version 2.23, by Roland McGrath et al.
    gdb --version: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
    
  • Tests fail with

    Tests fail with "unexpected fault address 0x117160" for aarch64

    Architecture: aarch64 Golang version: 1.8.3 OS: Fedora 26

    Same tests fail for other architectures randomly. Could not reproduce for those other arches though.

    go test -compiler gc -ldflags '' github.com/fortytw2/leaktest unexpected fault address 0x117160 fatal error: fault [signal SIGSEGV: segmentation violation code=0x2 addr=0x117160 pc=0x117160] goroutine 18 [running]: runtime.throw(0x147168, 0x5) /usr/lib/golang/src/runtime/panic.go:596 +0x70 fp=0x4420040eb0 sp=0x4420040e90 runtime.sigpanic() /usr/lib/golang/src/runtime/signal_unix.go:297 +0x224 fp=0x4420040f00 sp=0x4420040eb0 github.com/fortytw2/leaktest.CheckTimeout.func1() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest.go:75 +0x38 fp=0x4420040f20 sp=0x4420040f10 github.com/fortytw2/leaktest.TestCheck(0x4420078270) /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:73 +0xb0 fp=0x4420040fa0 sp=0x4420040f20 testing.tRunner(0x4420078270, 0x150138) /usr/lib/golang/src/testing/testing.go:657 +0x84 fp=0x4420040fc0 sp=0x4420040fa0 runtime.goexit() /usr/lib/golang/src/runtime/asm_arm64.s:981 +0x4 fp=0x4420040fc0 sp=0x4420040fc0 created by testing.(*T).Run /usr/lib/golang/src/testing/testing.go:697 +0x240 goroutine 1 [chan receive]: testing.(*T).Run(0x44200781a0, 0x147b6e, 0x9, 0x150138, 0x60) /usr/lib/golang/src/testing/testing.go:698 +0x260 testing.runTests.func1(0x44200781a0) /usr/lib/golang/src/testing/testing.go:882 +0x58 testing.tRunner(0x44200781a0, 0x442003fde0) /usr/lib/golang/src/testing/testing.go:657 +0x84 testing.runTests(0x4420086980, 0x1f1480, 0x2, 0x2, 0x14b00a) /usr/lib/golang/src/testing/testing.go:888 +0x260 testing.(*M).Run(0x442003ff28, 0x442003ff28) /usr/lib/golang/src/testing/testing.go:822 +0xe0 main.main() github.com/fortytw2/leaktest/_test/_testmain.go:44 +0xe0 goroutine 20 [sleep]: time.Sleep(0x3b9aca00) /usr/lib/golang/src/runtime/time.go:59 +0xe8 github.com/fortytw2/leaktest.glob..func1() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:25 +0x20 created by github.com/fortytw2/leaktest.TestCheck /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:71 +0xa4 goroutine 37 [chan receive]: github.com/fortytw2/leaktest.glob..func2() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:32 +0x48 created by github.com/fortytw2/leaktest.TestCheck /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:71 +0xa4 FAIL github.com/fortytw2/leaktest 2.078s

  • Add CheckTimeout function

    Add CheckTimeout function

    Allows for a configurable timeout without changing the previously exposed API :)

    closes https://github.com/fortytw2/leaktest/issues/5

    cc/ @lukechampine @bradleyfalzon

  • Memory Leak Not Detected When Leaking Goroutine

    Memory Leak Not Detected When Leaking Goroutine

    Given that golang is garbage-collected, I'm trying to understand instances when it would make sense to use this package. As such, I tried to simulate a memory leak.

    
    func TestGoRoutineLeak(t *testing.T) {
    
    	leaktest.Check(t)()
    	c := make (chan int,0)
    
    	go func () {
    		for _ = range c{
    
    		}
    
    	}()
    }
    

    This doesn't throw any errors? I was expecting it to throw an error since we're leaking a goroutine in plain sight. I'd love a pointer or two in the right direction here.

    Thanks :)

  • Tracks interesting goroutines by ID instead of stacktrace

    Tracks interesting goroutines by ID instead of stacktrace

    Previously the leak detection compared the goroutine stacks from the starting of a function with their stacks at the end to find new goroutines. But, when there are preexisting goroutines whose stacks change during the function, they are incorrectly detected as a leaked goroutine. This update compare goroutine IDs instead.

  • Prevent broken usage

    Prevent broken usage

    defer leaktest.Check(t) is surprisingly incorrect and does nothing. Lets investigate ways to detect when this happens and inform the caller, otherwise they may think that their code is leak free and dandy when it really isn't

  • Test timeouts

    Test timeouts

    leaktest does not play well with go test -timeout=5s. Regardless, waiting 5 seconds for goroutines to shut down seems excessive. My expectation was that goroutines would be checked as soon as the deferred function was called. I suggest adding an optional timeout argument to Check, or providing some other way of delaying the check. Note that if Check did not delay by default, you could implement this behavior yourself:

    func TestFoo(t testing.T) {
        checkFn := leaktest.Check(t)
        defer func() {
            time.Sleep(5 * time.Second)
            checkFn()
        }()
    }
    

    Furthermore, this would give you more fine-grained control over when the check executes. For example, you could wait on a channel before checking. The only reason I propose adding an optional timeout argument is because it seems like a common case.

  • README example is missing a ()

    README example is missing a ()

    Example in README should be

    func TestPool(t *testing.T) {
        defer leaktest.Check(t)()
    
        go func() {
            for {
                time.Sleep(time.Second)
            }
        }()
    }
    

    Notice the () after the } at the end of the go function -- Just a typo I am certain.

  • Ignore HTTP Keep-alives

    Ignore HTTP Keep-alives

    As per the change in Cockroach leaktest I'd like to ignore HTTP keep-alives in this utility.

    leaktest: ignore HTTP keep alives Tests that performed an HTTP connection using the default client had to have a special instruction to poke http.DefaultClient to disable keep alives to prevent the leaktester from failing. An upcoming change performs an HTTP connection with its own unconfigurable client, which would have required a bunch of special code just to make the leak tester pass. Instead, ignore stacks that are included during a connection in keep alive. This matches the code in net/http/main_test.go#afterTest.

    https://github.com/cockroachdb/cockroach/commit/e5e6dde6a58d551631640261f9e08c74c87ae917#diff-fdc7f75e12e2453349457c6742ff532f

  • refactor: extract 'LeakCheckConfiguration'

    refactor: extract 'LeakCheckConfiguration'

    👋 Hello—thank you for making this package, I was just about to start copying stuff out of net/http until I found it!

    One thing which would be convenient for me is the ability to specify a certain goroutine is "not interesting"—e.g. sometimes my code might be using some other package which starts some internal goroutine which by design runs forever. It would be nice if I could basically add it to the ignore list in the context of that particular test.

    This PR attempts to achieve that by factoring out the slice of 'known disinteresting' values to a configuration struct, which can then be customised arbitrarily. The existing surface of the library doesn't change, as the package level functions now delegate to a package global default configuration.

  • feat: add skip options in checks

    feat: add skip options in checks

    This PR introduces options to skip some goroutines based on their stack.

    Use case: I know something I depend on is leaking, or it is ok that something is leaking, and I want only to error on a precise leak.

  • Error parsing stack

    Error parsing stack

    I'm seeing a mysterious issue when I use leaktest in combination with a high -count flag, as in:

    go test -run . -failfast -count 100 --tags=unit ./...
        leaktest.go:83: leaktest: error parsing stack: "goro"
    FAIL
    

    The reported line number in leaktest.go is always the same. But the exact string it's failing to parse changes from run to run. e.g.

    leaktest.go:83: leaktest: error parsing stack: "gorouti"
    leaktest.go:83: leaktest: error parsing stack: "goroutine 99673 [select]:"
    

    It seems to happen more predictably when I run with high -count values.

    Any ideas what could be causing this?

    Here is my go build env:

    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/home/marshall/.cache/go-build"
    GOENV="/home/marshall/.config/go/env"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/home/marshall/code/go"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/usr/lib/go"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="x86_64-pc-linux-gnu-gcc"
    CXX="x86_64-pc-linux-gnu-g++"
    CGO_ENABLED="1"
    GOMOD=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build261227009=/tmp/go-build -gno-record-gcc-switches"
    
    
  • Plans for supporting t.Parallel

    Plans for supporting t.Parallel

    Hello!

    Thanks for this library, it'll definitely come in handy. Would you be able to document the reasons why t.Parallel isn't supported and what ideas you have for addressing the problem?

    Thanks

🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Jan 2, 2023
🐝 A Highly Performant and easy to use goroutine pool for Go
🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Sep 26, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

Dec 19, 2022
Lightweight Goroutine pool

grpool Lightweight Goroutine pool Clients can submit jobs. Dispatcher takes job, and sends it to first available worker. When worker is done with proc

Dec 6, 2022
errgroup with goroutine worker limits

neilotoole/errgroup neilotoole/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but limited to N goroutines. This is useful for inter

Dec 15, 2022
Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Dec 22, 2022
A goroutine pool for Go
A goroutine pool for Go

Tunny is a Golang library for spawning and managing a goroutine pool, allowing you to limit work coming from any number of goroutines with a synchrono

Dec 31, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Dec 28, 2022
Waiting group for collecting goroutine information.

在go语言waitGroup和errGroup都是用来控制goroutine的并发的方式,前者只能等待所有goroutine执行完成之后再执行Wait()函数后面的代码并且不

Dec 3, 2021
A simple and useful goroutine concurrent library.

Taskgroup A simple and useful goroutine concurrent library. Installation go get github.com/anthhub/taskgroup

May 19, 2021
Provides some convenient API, includes Goid(), AllGoid(), and LocalStorage, which is a goroutine's local storage, just like ThreadLocal in other languages.

routine 中文版 routine encapsulates and provides some easy-to-use, high-performance goroutine context access interfaces, which can help you access corout

Dec 30, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 9, 2023
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 2, 2023
A universal mechanism to manage goroutine lifecycles

A universal mechanism to manage goroutine lifecycles

Dec 29, 2022
goroutine pool in golang

goroutine pool in golang

Nov 1, 2021
A cross goroutine storage tool with very simple implementation and function.

Simple-goroutine-local is a cross goroutine storage tool with very simple implementation and function (the concept is similar to Java ThreadLocal). Ge

Jan 13, 2022
A lib for monitoring runtime goroutine stack

Overview A lib for monitoring runtime goroutine stack. Such as wait for goroutines to exit, leak detection, etc. Features context.Context first design

Oct 31, 2022
Goroutine Leak Detector

Leaktest Refactored, tested variant of the goroutine leak detector found in both net/http tests and the cockroachdb source tree. Takes a snapshot of r

Dec 26, 2022
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation

Package pool Package pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. Features

Jan 1, 2023
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Jan 2, 2023