This repository collects common concurrency patterns in Golang

Go Concurrency Patterns

This repository collects common concurrency patterns in Golang

Materials

Context:

Name Description
1-boring A hello world to goroutine
2-chan A hello world to go channel
3-generator A python-liked generator
4-fanin Fan in pattern
5-restore-sequence Restore sequence
6-select-timeout Add Timeout to a goroutine
7-quit-signal Quit signal
8-daisy-chan Daisy chan pattern
9-google1.0 Build a concurrent google search from the grown-up
10-google2.0 Build a concurrent google search from the grown-up
11-google2.1 Build a concurrent google search from the grown-up
12-google3.0 Build a concurrent google search from the grown-up
13-adv-pingpong A sample ping-pong table implemented in goroutine
14-adv-subscription Subscription
15-bounded-parallelism Bounded parallelism
16-context How to user context in HTTP client and server
17-ring-buffer-channel Ring buffer channel
18-worker-pool worker pool pattern
Owner
Kha Nguyen
I am an independent security enthusiast, AI engineer. I love Vim and Go. I use this corner to share tools that I work on.
Kha Nguyen
Similar Resources

Off heap golang memory pool

Stealthpool stealthpool provides a memory pool that allocates blocks off-heap that will NOT be tracked by the garbage collector. The name stealthpool

Dec 5, 2022

Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 9, 2023

Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 2, 2023

goroutine pool in golang

goroutine pool in golang

Nov 1, 2021

Routine - ThreadLocal for golang

routine 中文版 routine encapsulates and provides some easy-to-use, high-performance

Jan 1, 2023

Golang Implementation of Worker Pool/ Thread Pool

Golang Implementation of Worker Pool/ Thread Pool

Jun 18, 2022

Goworkers - Zero dependency Golang worker pool

Golang Worker Pool Zero dependency golang goroutines pool library. It is useful

Apr 28, 2022

Go-miningcore-pool - Miningcore Pool written in GOlang

Go-Miningcore-Pool (COMING SOON) Miningcore Pool written in GOlang 0x01 Configur

Apr 24, 2022

Worker - A Golang library that provides worker pools

Worker A Golang library that provides worker pools. Usage See *_test.go files. T

Apr 15, 2022
Comments
  • Different approach for ring buffer

    Different approach for ring buffer

    https://github.com/lotusirous/go-concurrency-patterns/blob/82c23134af1a6f4a847338e0eadfde940a35a608/17-ring-buffer-channel/main.go#L28

    This is a bug surely? Another routine could write between these two calls, and then this write would block.

    A safer approach would be to drop the current write AND one from the ring buffer. This would ensure that no code ever blocks, but you are still guaranteed new data (which IMO is a big use-case for ring buffers).

    This would ensure no back-pressure. It would make starvation technically possible very rarely by some complex timing and workload patterns if you have too many writers for the size of the ring buffer, but this is trivial to avoid by sizing the ring buffer appropriately.

    So as long as your ring buffer isn't too small, I think it would behave more like what people would expect.

  • timeout example is unsuitable

    timeout example is unsuitable

    https://github.com/lotusirous/go-concurrency-patterns/blob/2c165868b5befa1f6da1c9cb7f6db7980c63d005/6-select-timeout/main.go#L25-L35

    We always think timeout as the time that one service has on response. In this case, timeout 's meaning should be ** we don't receive any message from c channel for 5 seconds.

    so maybe the right code is :

    timeout := time.After(5 * time.Second)
    for {
           select {
           case s := <-c:
                   fmt.Println(s)
                   timeout = time.After(5 * time.Second) // update timeout time
           case <-timeout:
                   fmt.Println("channel has no response for 5 seconds")
                   return
           }
    }
    

    the origin code seems that you want to receive from every channel for same time, like linux cfs scheduler. so maybe in that case we should use ticker

    package main
    
    import (
            "fmt"
            "time"
    )
    
    func boring(id int) <-chan string {
            c := make(chan string)
            go func() {
                     c <- "" // skip the first time
                    for i := 0; ; i++ {
                            c <- fmt.Sprintf("%d, %s", id, time.Now().Format("2006-01-02 15:04:05"))
                            time.Sleep(1 * time.Second)
                    }
            }()
            return c
    }
    
    func main() {
            timeout := time.NewTicker(5 * time.Second)
            c1 := boring(1)
            c2 := boring(2)
            jobchannels := []<-chan string{c1, c2}
            i := 0
            for {
                    select {
                    case s := <-jobchannels[i]:
                            fmt.Println(s)
                    case <-timeout.C:
                            fmt.Printf("%d has talk for 5 secs\n", i+1)
                            i = (i + 1) % len(jobchannels)
                    }
            }
    }
    

    and it's output is

    
    1, 2021-03-17 11:27:03
    1, 2021-03-17 11:27:04
    1, 2021-03-17 11:27:05
    1, 2021-03-17 11:27:06
    1, 2021-03-17 11:27:07
    1 has talk for 5 secs
    
    2, 2021-03-17 11:27:08
    2, 2021-03-17 11:27:09
    2, 2021-03-17 11:27:10
    2, 2021-03-17 11:27:11
    2, 2021-03-17 11:27:12
    2 has talk for 5 secs
    1, 2021-03-17 11:27:08
    1, 2021-03-17 11:27:14
    1, 2021-03-17 11:27:15
    1, 2021-03-17 11:27:16
    ....
    
A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

Dec 31, 2022
gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Oct 31, 2022
Structured Concurrency in Go

nursery: structured concurrency in Go RunConcurrently( // Job 1 func(context.Context, chan error) { time.Sleep(time.Millisecond * 10)

Dec 27, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Dec 28, 2022
Pengenalan Concurrency dan Parallel Programming
Pengenalan Concurrency dan Parallel Programming

Golang Goroutine Sumber Tutorial: Udemy Slide Pengenalan Concurrency dan Parallel Programming Pengenalan Parallel Programming Saat ini kita hidup dima

Nov 5, 2021
Simple in-memory job queue for Golang using worker-based dispatching

artifex Simple in-memory job queue for Golang using worker-based dispatching Documentation here: https://godoc.org/github.com/mborders/artifex Cron jo

Dec 24, 2022
CyclicBarrier golang implementation

cyclicbarrier CyclicBarrier is a synchronizer that allows a set of goroutines to wait for each other to reach a common execution point, also called a

Nov 30, 2022
TryLock support on read-write lock for Golang

go-trylock TryLock support on read-write lock for Golang Interface go-trylock implements sync.Locker. Have same interfaces with sync.RWMutex Documenta

Sep 26, 2022
Fast resizable golang semaphore primitive

semaphore Fast resizable golang semaphore based on CAS allows weighted acquire/release; supports cancellation via context; allows change semaphore lim

Dec 13, 2022
Golang simple thread pool implementation

Golang Threadpool implementation Scalable threadpool implementation using Go to handle the huge network trafic. Install go get github.com/shettyh/thre

Dec 12, 2022