Golang simple thread pool implementation

Golang Threadpool implementation

Build Status codecov GoDoc Go Report Card

Scalable threadpool implementation using Go to handle the huge network trafic.

Install

go get github.com/shettyh/threadpool

Usage

Threadpool

  • Implement Runnable interface for tha task that needs to be executed. For example

    type MyTask struct { }
     
    func (t *MyTask) Run(){
      // Do your task here
    }
     
    
  • Create instance of ThreadPool with number of workers required and the task queue size

    pool := threadpool.NewThreadPool(200,1000000)
    
  • Create Task and execute

    task:=&MyTask{}
    err := pool.Execute(task)
    
  • Using Callable task

    type MyTaskCallable struct { }
    
    func (c *MyTaskCallable) Call() interface{} {
      //Do task 
      return result
    }
    
    //Execute callable task
    task := &MyTaskCallable{}
    future, err := pool.ExecuteFuture(task)
    
    //Check if the task is done
    isDone := future.IsDone() // true/false
    
    //Get response , blocking call
    result := future.Get()
    
    
  • Close the pool

    pool.Close()
    

Scheduled threadpool

  • Create instance of ScheduledThreadPool with number of workers required
    schedulerPool:= threadpool.NewScheduledThreadPool(10)
    
  • Create Task and schedule
    task:=&MyTask{}
    pool.ScheduleOnce(task, time.Second*20) // Time delay is in seconds only as of now
    
  • Close the pool
    pool.Close()
    
Owner
Similar Resources

Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 2, 2023

goroutine pool in golang

goroutine pool in golang

Nov 1, 2021

Goworkers - Zero dependency Golang worker pool

Golang Worker Pool Zero dependency golang goroutines pool library. It is useful

Apr 28, 2022

🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants ζ˜―δΈ€δΈͺι«˜ζ€§θƒ½δΈ”δ½ŽζŸθ€—ηš„ goroutine 池。

🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants ζ˜―δΈ€δΈͺι«˜ζ€§θƒ½δΈ”δ½ŽζŸθ€—ηš„ goroutine 池。

A goroutine pool for Go English | πŸ‡¨πŸ‡³ δΈ­ζ–‡ πŸ“– Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Jan 2, 2023

🐝 A Highly Performant and easy to use goroutine pool for Go

🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Sep 26, 2022

gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Oct 31, 2022

Lightweight Goroutine pool

grpool Lightweight Goroutine pool Clients can submit jobs. Dispatcher takes job, and sends it to first available worker. When worker is done with proc

Dec 6, 2022

Unlimited job queue for go, using a pool of concurrent workers processing the job queue entries

kyoo: A Go library providing an unlimited job queue and concurrent worker pools About kyoo is the phonetic transcription of the word queue. It provide

Dec 21, 2022

Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Dec 22, 2022
Comments
  • Queue size never get full

    Queue size never get full

    Hey,

    I was trying this lib and found out a small issue. The queue buffer size doesn't work as expected for low values. For example I've some tasks that I want to run but I don't want to allow more than let's say 2 jobs on the queue. After that I want to refuse and return an error saying that the queue is already full.

    So on the first place, the current implementation doesn't allow me to return an error if the queue gets full, since both methods Execute and ExecuteFuture just push the job to the queue channel and if the channel gets full the caller would just block.

    Besides that, the queue never gets full, I created a threadpool with a job queue with the size 1 and called multiple times the Execute/ExecuteFuture but the queue never gets full. The dispatch method is always reading from the queue and fanning out a go routine with the job. Instead of holding the jobs on the queue, you are fanning out go routines holding the job and waiting for a worker.

    Thoughts?

  • Why bother with a chan chan of workers?

    Why bother with a chan chan of workers?

    https://github.com/shettyh/threadpool/blob/b99fd8aaa9451aabc3d737e614c9a24a44d2a84c/threadpool.go#L15

    Wouldn't it be simpler to just have workers pull jobs off the job queue instead of a double redirect / dispatch? (get job, find worker, queue job).

Golang simple thread pool implementation

Golang Threadpool implementation Scalable threadpool implementation using Go to handle the huge network trafic. Install go get github.com/shettyh/thre

Dec 12, 2022
Go-ldap-pool - A simple connection pool for go-ldap

Basic connection pool for go-ldap This little library use the go-ldap library an

Dec 17, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool δΈ­ζ–‡θ―΄ζ˜Ž Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

Dec 19, 2022
Go-miningcore-pool - Miningcore Pool written in GOlang

Go-Miningcore-Pool (COMING SOON) Miningcore Pool written in GOlang 0x01 Configur

Apr 24, 2022
Work pool channlege - An url hash retriever worker pool for getting hash digest for a collection of urls

Code challenge The aim of this project is to provide an url hash retriever worke

Feb 16, 2022
Go simple async worker pool
Go simple async worker pool

??‍?? worker-pool Go simple async worker pool. ?? ABOUT Worker pool is a software design pattern for achieving concurrency of task execution. Maintain

Sep 26, 2022
Deadly simple worker pool

go-worker-pool Deadly simple worker pool Usage package main import ( "errors" workerpool "github.com/zelenin/go-worker-pool" "log" "time" ) func

Dec 10, 2021
Gopool - A simple goroutione pool

GoPool gopool is simple goroutione pool

Jun 10, 2022
Off heap golang memory pool

Stealthpool stealthpool provides a memory pool that allocates blocks off-heap that will NOT be tracked by the garbage collector. The name stealthpool

Dec 5, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Jan 9, 2023