concurrency utilities

concurrent

Sourcegraph GoDoc Build Status codecov rcard License

  • concurrent.Map: backport sync.Map for go below 1.9
  • concurrent.Executor: goroutine with explicit ownership and cancellable

concurrent.Map

because sync.Map is only available in go 1.9, we can use concurrent.Map to make code portable

m := concurrent.NewMap()
m.Store("hello", "world")
elem, found := m.Load("hello")
// elem will be "world"
// found will be true

concurrent.Executor

executor := concurrent.NewUnboundedExecutor()
executor.Go(func(ctx context.Context) {
    everyMillisecond := time.NewTicker(time.Millisecond)
    for {
        select {
        case <-ctx.Done():
            fmt.Println("goroutine exited")
            return
        case <-everyMillisecond.C:
            // do something
        }
    }
})
time.Sleep(time.Second)
executor.StopAndWaitForever()
fmt.Println("executor stopped")

attach goroutine to executor instance, so that we can

  • cancel it by stop the executor with Stop/StopAndWait/StopAndWaitForever
  • handle panic by callback: the default behavior will no longer crash your application
Owner
Modern Go Programming
Better developer experience
Modern Go Programming
https://github.com/modern-go/concurrent
Comments
  • codecov in .travis.yml - security issue

    codecov in .travis.yml - security issue

    Good day, https://codecov.io/bash recently outputted a security issue. Details can be found at https://about.codecov.io/security-update/

    The following file and code are impacted. Please follow the recommended actions within the link above to secure your env and code integrity. file: .travis.yml Code: after_success:

    bash <(curl -s https://codecov.io/bash)

  • heads up: codecov.io security incident - https://about.codecov.io/security-update/

    heads up: codecov.io security incident - https://about.codecov.io/security-update/

    Hi there.

    This might be an unusual "issue" beeing reported.

    There has been a security incident in codecov.io with the bash-uploader script (see [1] for details) which potentially exposed secrets to 3rd parties.

    It seems you are using the referenced bash uploader in your .travis.yml file. I wanted to draw your attention to this incident in case you missed it.

    It would be great if you could verify that no code has been altered and check the impact of this security incident on your repository.

    Regards, Robert

    [1] https://about.codecov.io/security-update/

  • Get rid of `context.TODO()`

    Get rid of `context.TODO()`

    Our code analysis is flagging the use of context.TODO() in NewUnboundedExecutor.

    What is the intent here? To pass a context to the constructor or if not, should context.TODO() be replaced by context.Background()?

  • Add a copyright / notice file

    Add a copyright / notice file

    This project doesn't appear to include any copyright information or a NOTICE file. Can you add one or update the Apache License with this information? This is desired to comply with the open source license conditions.

  • All tests are failing, if using gollvm

    All tests are failing, if using gollvm

    Hello.

    Here is my setup

    $ go get -v -x -u github.com/modern-go/concurrent cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git config remote.origin.url github.com/modern-go/concurrent (download) cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git pull --ff-only cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git submodule update --init --recursive cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git show-ref cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git submodule update --init --recursive WORK=/tmp/go-build480857928 github.com/modern-go/concurrent mkdir -p $WORK/b001/ cd $WORK /usr/local/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true cd /home/oceanfish81/go/src/github.com/modern-go/concurrent /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=github.com/modern-go/concurrent -o $WORK/b001/go.o -I $WORK/b001/importcfgroot ./executor.go ./go_above_19.go ./log.go ./unbounded_executor.go echo ' .section .go.buildid,"e"' >> $WORK/b001/_buildid.s echo ' .byte 0x76,0x77,0x37,0x43,0x53,0x34,0x5a,0x72' >> $WORK/b001/_buildid.s echo ' .byte 0x5a,0x42,0x6f,0x4b,0x31,0x2d,0x30,0x6f' >> $WORK/b001/_buildid.s echo ' .byte 0x69,0x53,0x2d,0x7a,0x2f,0x76,0x77,0x37' >> $WORK/b001/_buildid.s echo ' .byte 0x43,0x53,0x34,0x5a,0x72,0x5a,0x42,0x6f' >> $WORK/b001/_buildid.s echo ' .byte 0x4b,0x31,0x2d,0x30,0x6f,0x69,0x53,0x2d' >> $WORK/b001/_buildid.s echo ' .byte 0x7a' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b001/_buildid.s echo '' >> $WORK/b001/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I $WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D GOPKGPATH=github.x2ecom..z2fmodern..z2dgo..z2fconcurrent -m64 $WORK/b001/_buildid.s ar rcD $WORK/b001/pkg.a $WORK/b001/go.o $WORK/b001/_buildid.o /usr/local/tools/buildid -w $WORK/b001/pkg.a # internal cp $WORK/b001/pkg.a /home/oceanfish81/.cache/go-build/35/35fa2bf6472674cdf2e146f2fab7911cd73474608b6a07d5790605cd8c9d8a3b-d # internal mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/ mv $WORK/b001/pkg.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a rm -r $WORK/b001/

    Checking if there wouldn't be any errors:

    $ go install -i -v -x github.com/modern-go/concurrent Checking an enforced build process: $ go build -i -v -x github.com/modern-go/concurrent WORK=/tmp/go-build024139235 Navigating into my source folder: $ cd ~/go/src/github.com/modern-go/concurrent

    And then I got two tests failing:

    $ go test map_test.go -x -v && go test unbounded_executor_test.go -v -x WORK=/tmp/go-build608777315 mkdir -p $WORK/b001/ mkdir -p $WORK/b009/ mkdir -p $WORK/b008/ cat >$WORK/b008/vet.cfg << 'EOF' # internal { "ID": "command-line-arguments_test", "Compiler": "gccgo", "Dir": "/home/oceanfish81/go/src/github.com/modern-go/concurrent", "ImportPath": "command-line-arguments_test", "GoFiles": [ "/home/oceanfish81/go/src/github.com/modern-go/concurrent/map_test.go" > ], "NonGoFiles": [], "ImportMap": { "github.com/modern-go/concurrent": "github.com/modern-go/concurrent", "testing": "testing" }, "PackageFile": { "github.com/modern-go/concurrent": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a" }, "Standard": { "testing": true }, "PackageVetx": { "github.com/modern-go/concurrent": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d" }, "VetxOnly": false, "VetxOutput": "$WORK/b008/vet.out", "SucceedOnTypecheckFailure": false } EOF cd /home/oceanfish81/go/src/github.com/modern-go/concurrent TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg === RUN TestMap_Load --- PASS: TestMap_Load (0.00s) PASS ok command-line-arguments (cached) rm -r $WORK/b001/ WORK=/tmp/go-build599881499 mkdir -p $WORK/b001/ mkdir -p $WORK/b011/ mkdir -p $WORK/b008/ cd $WORK /usr/local/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true mkdir -p $WORK/b008/importcfgroot/github.com/modern-go ln -s /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a $WORK/b008/importcfgroot/github.com/modern-go/libconcurrent.a cd /home/oceanfish81/go/src/github.com/modern-go/concurrent /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=command-line-arguments_test -fgo-relative-import-path=_/home/oceanfish81/go/src/github.com/modern-go/concurrent -o $WORK/b008/go.o -I $WORK/b008/importcfgroot ./unbounded_executor_test.go echo ' .section .go.buildid,"e"' >> $WORK/b008/_buildid.s echo ' .byte 0x64,0x49,0x4c,0x30,0x61,0x6e,0x54,0x46' >> $WORK/b008/_buildid.s echo ' .byte 0x37,0x70,0x4b,0x68,0x47,0x70,0x50,0x32' >> $WORK/b008/_buildid.s echo ' .byte 0x76,0x79,0x79,0x30,0x2f,0x64,0x49,0x4c' >> $WORK/b008/_buildid.s echo ' .byte 0x30,0x61,0x6e,0x54,0x46,0x37,0x70,0x4b' >> $WORK/b008/_buildid.s echo ' .byte 0x68,0x47,0x70,0x50,0x32,0x76,0x79,0x79' >> $WORK/b008/_buildid.s echo ' .byte 0x30' >> $WORK/b008/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b008/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b008/_buildid.s echo '' >> $WORK/b008/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I $WORK/b008/ -c -o $WORK/b008/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D GOPKGPATH=command..z2dline..z2darguments_test -m64 $WORK/b008/_buildid.s ar rcD $WORK/b008/pkg.a $WORK/b008/go.o $WORK/b008/_buildid.o /usr/local/tools/buildid -w $WORK/b008/pkg.a # internal cp $WORK/b008/pkg.a /home/oceanfish81/.cache/go-build/1f/1fd3390c5dd92a008d2092c605ef674dc8ab4e8cfe0b010f939d168507b69f1d-d # internal cat >$WORK/b008/vet.cfg << 'EOF' # internal { "ID": "command-line-arguments_test", "Compiler": "gccgo", "Dir": "/home/oceanfish81/go/src/github.com/modern-go/concurrent", "ImportPath": "command-line-arguments_test", "GoFiles": [ "/home/oceanfish81/go/src/github.com/modern-go/concurrent/unbounded_executor_test.go" > ], "NonGoFiles": [], "ImportMap": { "context": "context", "fmt": "fmt", "github.com/modern-go/concurrent": "github.com/modern-go/concurrent", "time": "time" }, "PackageFile": { "github.com/modern-go/concurrent": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a" }, "Standard": { "context": true, "fmt": true, "time": true }, "PackageVetx": { "github.com/modern-go/concurrent": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d" }, "VetxOnly": false, "VetxOutput": "$WORK/b008/vet.out", "SucceedOnTypecheckFailure": false } EOF TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg mkdir -p $WORK/b001/importcfgroot ln -s $WORK/b008/pkg.a $WORK/b001/importcfgroot/libcommand-line-arguments_test.a cd $WORK/b001/ /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o ./go.o -I ./importcfgroot ./_testmain.go echo ' .section .go.buildid,"e"' >> $WORK/b001/_buildid.s echo ' .byte 0x68,0x5f,0x4a,0x52,0x77,0x48,0x41,0x50' >> $WORK/b001/_buildid.s echo ' .byte 0x72,0x4a,0x42,0x65,0x56,0x37,0x32,0x51' >> $WORK/b001/_buildid.s echo ' .byte 0x70,0x6c,0x43,0x79,0x2f,0x68,0x5f,0x4a' >> $WORK/b001/_buildid.s echo ' .byte 0x52,0x77,0x48,0x41,0x50,0x72,0x4a,0x42' >> $WORK/b001/_buildid.s echo ' .byte 0x65,0x56,0x37,0x32,0x51,0x70,0x6c,0x43' >> $WORK/b001/_buildid.s echo ' .byte 0x79' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b001/_buildid.s echo '' >> $WORK/b001/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I ./ -c -o ./_buildid.o -D GOOS_linux -D GOARCH_amd64 -m64 ./_buildid.s ar rcD ./pkg.a ./go.o ./_buildid.o /usr/local/tools/buildid -w $WORK/b001/pkg.a # internal cp $WORK/b001/pkg.a /home/oceanfish81/.cache/go-build/07/072aa11d607b29519a08949c3fb6e70f5c63af6047da44221df21a9212ad5822-d # internal cat >$WORK/b001/importcfg.link << 'EOF' # internal packagefile command-line-arguments.test=$WORK/b001/pkg.a packagefile command-line-arguments_test=$WORK/b008/pkg.a packagefile github.com/modern-go/concurrent=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a EOF cd . /usr/local/bin/llvm-goc -o $WORK/b001/concurrent.test "-Wl,-(" -m64 -Wl,--whole-archive $WORK/b001/pkg.a $WORK/b008/pkg.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a -Wl,--no-whole-archive "-Wl,-)" -Wl,--build-id=0x66657a6d737562503939456e6e6161584575326b2f685f4a5277484150724a426556373251706c43792f34505f7041394f4e59306b7071657952414141662f66657a6d737562503939456e6e6161584575326b $WORK/b001/concurrent.test -test.testlogfile=$WORK/b001/testlog.txt -test.timeout=10m0s -test.v=true === RUN ExampleUnboundedExecutor_Go --- PASS: ExampleUnboundedExecutor_Go (1.00s) === RUN ExampleUnboundedExecutor_StopAndWaitForever --- PASS: ExampleUnboundedExecutor_StopAndWaitForever (1.10s) === RUN ExampleUnboundedExecutor_Go_panic --- FAIL: ExampleUnboundedExecutor_Go_panic (1.00s) got: concurrent_test.willPanic want: github.com/modern-go/concurrent_test.willPanic FAIL FAIL command-line-arguments 3.541s rm -r $WORK/b001/ FAIL

    Ivan

Related tags
Performance go golang concurrency concurrent
Go-concurrency-patterns - Sample concurrency patterns with Goroutines

About This sample project provides some concurrency pattern examples in Go using

Feb 21, 2022
concurrency utilities

concurrent concurrent.Map: backport sync.Map for go below 1.9 concurrent.Executor: goroutine with explicit ownership and cancellable concurrent.Map be

Dec 28, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Jan 1, 2023
A simple logging interface that supports cross-platform color and concurrency.
 A simple logging interface that supports cross-platform color and concurrency.

WLog Package wlog creates simple to use UI structure. The UI is used to simply print to the screen. There a wrappers that will wrap each other to crea

Sep 26, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Dec 31, 2022
Code Generation for Functional Programming, Concurrency and Generics in Golang

goderive goderive derives mundane golang functions that you do not want to maintain and keeps them up to date. It does this by parsing your go code fo

Dec 25, 2022
A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

Dec 31, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

Dec 19, 2022
gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Oct 31, 2022
Structured Concurrency in Go

nursery: structured concurrency in Go RunConcurrently( // Job 1 func(context.Context, chan error) { time.Sleep(time.Millisecond * 10)

Dec 27, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Dec 28, 2022
Pholcus is a distributed high-concurrency crawler software written in pure golang
 Pholcus is a distributed high-concurrency crawler software written in pure golang

Pholcus Pholcus（幽灵蛛）是一款纯 Go 语言编写的支持分布式的高并发爬虫软件，仅用于编程学习与研究。 它支持单机、服务端、客户端三种运行模式，拥有Web、GUI、命令行三种操作界面；规则简单灵活、批量任务并发、输出方式丰富（mysql/mongodb/kafka/csv/excel等

Dec 30, 2022
Helper library for full uint64 randomness, pool backed for efficient concurrency

fastrand64-go Helper library for full uint64 randomness, pool backed for efficient concurrency Inspired by https://github.com/valyala/fastrand which i

Dec 5, 2021
dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies
 dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

Dec 19, 2022
Concurrency in Go video course with in depth explanations & examples
 Concurrency in Go video course with in depth explanations & examples

Concurrency in Go Summary Coding Examples Introduction to Concurrency Go Routines Channels Select Concurrency Patterns Atomics Wait Groups - sync.Wait

Dec 26, 2022
yakv is a simple, in-memory, concurrency-safe key-value store for hobbyists.
 yakv is a simple, in-memory, concurrency-safe key-value store for hobbyists.

yakv (yak-v. (originally intended to be "yet-another-key-value store")) is a simple, in-memory, concurrency-safe key-value store for hobbyists. yakv provides persistence by appending transactions to a transaction log and restoring data from the transaction log on startup.

Feb 24, 2022
This repository collects common concurrency patterns in Golang

Go Concurrency Patterns This repository collects common concurrency patterns in Golang Materials Concurrency is not parallelism Go Concurrency Pattern

Jan 9, 2023
Concurrency patterns in Go

Concurrency patterns in Go

Dec 28, 2022
File Processor in Concurrency Pattern using Golang goroutine.
 File Processor in Concurrency Pattern using Golang goroutine.

File Processor in Concurrency Pattern Implement a file processor solution in concurrency pattern using Golang goroutine. Get Started Run docker-compos

Sep 16, 2022
Concurrency Lab 1 Go Example
 Concurrency Lab 1 Go Example

Concurrency Lab 1 If you're stuck look at examples on Go by Example Using the lab sheet There are two ways to use the lab sheet, you can either: creat

Oct 11, 2021