Concurrency patterns in Go

Go Concurrency Patterns

This repository collects common concurrency patterns in Golang

Materials

Context:

Name Description Playground
1-boring A hello world to goroutine play
2-chan A hello world to go channel play
3-generator A python-liked generator play
4-fanin Fan in pattern play
5-restore-sequence Restore sequence play
6-select-timeout Add Timeout to a goroutine play
7-quit-signal Quit signal play
8-daisy-chan Daisy chan pattern play
9-google1.0 Build a concurrent google search from the ground-up play
10-google2.0 Build a concurrent google search from the ground-up play
11-google2.1 Build a concurrent google search from the ground-up play
12-google3.0 Build a concurrent google search from the ground-up play
13-adv-pingpong A sample ping-pong table implemented in goroutine play
14-adv-subscription Subscription play
15-bounded-parallelism Bounded parallelism play
16-context How to user context in HTTP client and server play
17-ring-buffer-channel Ring buffer channel play
18-worker-pool worker pool pattern play
Owner
Kha Nguyen
I am an independent security enthusiast, AI engineer. I love Vim and Go. I use this corner to share tools that I work on.
Kha Nguyen
Similar Resources

Practical concurrency guide in Go, communication by channels, patterns

Go Concurrency Guide This guide is built on top of the some examples of the book Go Concurrency in Go and Go Programming Language Race Condition and D

Dec 28, 2022

A faster RWLock primitive in Go, 2-3 times faster than RWMutex. A Go implementation of concurrency control algorithm in paper Left-Right - A Concurrency Control Technique with Wait-Free Population Oblivious Reads

Go Left Right Concurrency A Go implementation of the left-right concurrency control algorithm in paper Left-Right - A Concurrency Control Technique w

Jan 6, 2023

Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, XML document namespaces, and tree magic for mounted volumes, generated from the XDG shared-mime-info database.

mimemagic Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, xml document namespaces, and tree ma

Nov 3, 2022

:triangular_ruler: Create beautiful generative image patterns from a string in golang.

:triangular_ruler: Create beautiful generative image patterns from a string in golang.

geopattern Create beautiful generative image patterns from a string in golang. Go port of Jason Long's awesome GeoPattern library. Read geopattern's d

Dec 29, 2022

Resiliency patterns for golang

go-resiliency Resiliency patterns for golang. Based in part on Hystrix, Semian, and others. Currently implemented patterns include: circuit-breaker (i

Jan 3, 2023

Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

Oct 28, 2022

A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

Jan 4, 2023

Lightweight static analysis for many languages. Find bug variants with patterns that look like source code.

Lightweight static analysis for many languages. Find bugs and enforce code standards. Semgrep is a fast, open-source, static analysis tool that finds

Jan 9, 2023

a Framework for creating mesh networks using technologies and design patterns of Erlang/OTP in Golang

a Framework for creating mesh networks using technologies and design patterns of Erlang/OTP in Golang

Ergo Framework Implementation of Erlang/OTP in Golang. Up to x5 times faster than original Erlang/OTP. The easiest drop-in replacement for your hot no

Jan 5, 2023

Censors or hides shell / Bash / console output based on defined patterns - great for hiding secrets in demos!

Censors or hides shell / Bash / console output based on defined patterns - great for hiding secrets in demos!

censor-shell Installation go install Usage Make the file ~/.censor-shell as an INI file with the following content: [nameofmyreplacement] pattern = b

Nov 11, 2022

23 design patterns of GoF

GoF 设计模式 GoF所提出的23种设计模式主要基于以下面向对象设计原则: 对接口编程而不是对实现编程 优先使用对象组合而不是继承 23种设计模式分为三大类:创建型模式(Creational Patterns)、结构型模式(Structural Patterns)、行为型模式(Behavioral

Nov 29, 2022

a Framework for creating microservices using technologies and design patterns of Erlang/OTP in Golang

a Framework for creating microservices using technologies and design patterns of Erlang/OTP in Golang

Technologies and design patterns of Erlang/OTP have been proven over the years. Now in Golang. Up to x5 times faster than original Erlang/OTP in terms

Dec 28, 2022

a Go code to detect leaks in JS files via regex patterns

a Go code to detect leaks in JS files via regex patterns

Nov 13, 2022

Short examples of common anti-patterns in Go Web Applications.

Go Web App Anti-Patterns This repository contains short examples of common anti-patterns in Go Web Applications. For complete description, see Common

Dec 31, 2022

Example patterns for distributed service

Cloud Native Tulisan ini akan mengulas hasil pembelajaran dari beberapa sumber #learn-from-books. Dan terinspirasi dari obrolan The Pursuit of Product

Jan 30, 2022

YAML and Golang implementations of common Kubernetes patterns.

Kubernetes Patterns Types Patterns Foundational Patterns Behavioral Patterns Structural Patterns Configuration Patterns Usage To run, simply do go run

Aug 8, 2022

Redis inventory is a tool to analyse Redis memory usage by key patterns and displaying it hierarchically

Redis inventory is a tool to analyse Redis memory usage by key patterns and displaying it hierarchically

Redis inventory is a tool to analyse Redis memory usage by key patterns and displaying it hierarchically. The name is inspired by "Disk Inventory X" tool doing similar analysis for disk usage.

Dec 11, 2022
Comments
  • Different approach for ring buffer

    Different approach for ring buffer

    https://github.com/lotusirous/go-concurrency-patterns/blob/82c23134af1a6f4a847338e0eadfde940a35a608/17-ring-buffer-channel/main.go#L28

    This is a bug surely? Another routine could write between these two calls, and then this write would block.

    A safer approach would be to drop the current write AND one from the ring buffer. This would ensure that no code ever blocks, but you are still guaranteed new data (which IMO is a big use-case for ring buffers).

    This would ensure no back-pressure. It would make starvation technically possible very rarely by some complex timing and workload patterns if you have too many writers for the size of the ring buffer, but this is trivial to avoid by sizing the ring buffer appropriately.

    So as long as your ring buffer isn't too small, I think it would behave more like what people would expect.

  • timeout example is unsuitable

    timeout example is unsuitable

    https://github.com/lotusirous/go-concurrency-patterns/blob/2c165868b5befa1f6da1c9cb7f6db7980c63d005/6-select-timeout/main.go#L25-L35

    We always think timeout as the time that one service has on response. In this case, timeout 's meaning should be ** we don't receive any message from c channel for 5 seconds.

    so maybe the right code is :

    timeout := time.After(5 * time.Second)
    for {
           select {
           case s := <-c:
                   fmt.Println(s)
                   timeout = time.After(5 * time.Second) // update timeout time
           case <-timeout:
                   fmt.Println("channel has no response for 5 seconds")
                   return
           }
    }
    

    the origin code seems that you want to receive from every channel for same time, like linux cfs scheduler. so maybe in that case we should use ticker

    package main
    
    import (
            "fmt"
            "time"
    )
    
    func boring(id int) <-chan string {
            c := make(chan string)
            go func() {
                     c <- "" // skip the first time
                    for i := 0; ; i++ {
                            c <- fmt.Sprintf("%d, %s", id, time.Now().Format("2006-01-02 15:04:05"))
                            time.Sleep(1 * time.Second)
                    }
            }()
            return c
    }
    
    func main() {
            timeout := time.NewTicker(5 * time.Second)
            c1 := boring(1)
            c2 := boring(2)
            jobchannels := []<-chan string{c1, c2}
            i := 0
            for {
                    select {
                    case s := <-jobchannels[i]:
                            fmt.Println(s)
                    case <-timeout.C:
                            fmt.Printf("%d has talk for 5 secs\n", i+1)
                            i = (i + 1) % len(jobchannels)
                    }
            }
    }
    

    and it's output is

    
    1, 2021-03-17 11:27:03
    1, 2021-03-17 11:27:04
    1, 2021-03-17 11:27:05
    1, 2021-03-17 11:27:06
    1, 2021-03-17 11:27:07
    1 has talk for 5 secs
    
    2, 2021-03-17 11:27:08
    2, 2021-03-17 11:27:09
    2, 2021-03-17 11:27:10
    2, 2021-03-17 11:27:11
    2, 2021-03-17 11:27:12
    2 has talk for 5 secs
    1, 2021-03-17 11:27:08
    1, 2021-03-17 11:27:14
    1, 2021-03-17 11:27:15
    1, 2021-03-17 11:27:16
    ....
    
Practical concurrency guide in Go, communication by channels, patterns

Go Concurrency Guide This guide is built on top of the some examples of the book Go Concurrency in Go and Go Programming Language Race Condition and D

Dec 28, 2022
23 design patterns of GoF

GoF 设计模式 GoF所提出的23种设计模式主要基于以下面向对象设计原则: 对接口编程而不是对实现编程 优先使用对象组合而不是继承 23种设计模式分为三大类:创建型模式(Creational Patterns)、结构型模式(Structural Patterns)、行为型模式(Behavioral

Nov 29, 2022
Binaryscarf generates double-knitting patterns for some corpus of input text.

binaryscarf binaryscarf generates double-knit patterns for some corpus of input text. The layout follows the same style as described here. Output is s

Dec 31, 2022
Go-design-pattern-examples - Golang implementations of common design patterns

Design Patterns Golang implementations of common design patterns Build project T

Oct 29, 2022
Go Design Patterns
Go Design Patterns

Creational patterns provide out-of-the-box objects for users instead relying on the user to know how to build the object which in some cases could be complex.

Feb 8, 2022
Patternfinder - Find patterns in http output based on regex string. Display occurences

Patternfinder Find patterns in HTTP output based on regex string. Display occurr

Feb 18, 2022
Understand go concurrency
Understand  go concurrency

CONTENT (click to expand or hide) What is Concurrency? Why we need to think about Concurrency? What is a Process? What is a Thread? Thread_States C10k

Jul 11, 2022
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.

LogAnalyzer Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory

May 31, 2022
This repository collects common concurrency patterns in Golang

Go Concurrency Patterns This repository collects common concurrency patterns in Golang Materials Concurrency is not parallelism Go Concurrency Pattern

Jan 9, 2023
Concurrency patterns in Go

Concurrency patterns in Go

Dec 28, 2022