A pool that spins up a given number of processes in advance and attaches stdin and stdout when needed

ExecPool

License Build Coverage Status Go Report Card GoDoc Release Awesome

Why? Sometimes when you implement Go services you have to integrate with third party command line tools. You can do it by simply using exec.Command but the problem with this approach is that your service has to wait while a new process is being loaded into the memory and started. Sometimes this can drastically increase the latency of your service. ExecPool works similarly to FastCGI but it can wrap any regular process. It spins up a given number of processes in advance and when it's time to handle a request from the user your service just attaches stdin to an existing process from the pool. Basically the execpool helps you trade memory for latency.

Usage

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"os/exec"
	"strings"

	"github.com/hexdigest/execpool"
)

func main() {
	cmd := exec.Command("grep", "none")

	//spin up 100 processes of grep
	pool, err := execpool.New(cmd, 100)

	rc := pool.Exec(strings.NewReader("this makes sense\nthis is nonesense"))
	b, err := ioutil.ReadAll(rc)
	if err != nil {
		log.Fatalf("failed to read from stdout: %v", err)
	}

	// this is nonesense
	fmt.Println(string(b))
}

Benchmark

This benchmark compares "standard" approach with exec.Command and execpool.Pool by running grep 100 times. For heavier processes you can expect a bigger difference.

make benchmark
goos: darwin
goarch: amd64
pkg: github.com/hexdigest/execpool
cpu: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
BenchmarkNew-8               100            941753 ns/op
BenchmarkCmd-8               100           2386990 ns/op
Owner
Max Chechel
Program or be programmed
Max Chechel
Similar Resources

Developed Code for the Fulfillment of the technical task given by Appointy using Go

INSTAGRAM BACKEND API BY KUNAL KULKARNI Table of Contents About The Project Built With Getting Started Prerequisites Installation Usage License Contac

Oct 10, 2021

Coding assessment to create Todo app given by Percipia

Coding assessment to create Todo app given by Percipia

Oct 20, 2021

A Github action to check if IDT could synthesize a given DNA sequence.

dna-is-synthesizable A github action to check if a part is synthesizable from a given Genbank file. dna-is-synthesizable is a Github Action that recei

Oct 28, 2021

Program to solve sudokus given in JSON Format, written in Go

Program to solve sudokus given in JSON Format, written in Go

May 2, 2022

Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Apr 21, 2022

[TOOL, CLI] - Filter and examine Go type structures, interfaces and their transitive dependencies and relationships. Export structural types as TypeScript value object or bare type representations.

typex Examine Go types and their transitive dependencies. Export results as TypeScript value objects (or types) declaration. Installation go get -u gi

Dec 6, 2022

:chart_with_upwards_trend: Monitors Go MemStats + System stats such as Memory, Swap and CPU and sends via UDP anywhere you want for logging etc...

Package stats Package stats allows for gathering of statistics regarding your Go application and system it is running on and sent them via UDP to a se

Nov 10, 2022

James is your butler and helps you to create, build, debug, test and run your Go projects

James is your butler and helps you to create, build, debug, test and run your Go projects

go-james James is your butler and helps you to create, build, debug, test and run your Go projects. When you often create new apps using Go, it quickl

Oct 8, 2022

GoThanks automatically stars Go's official repository and your go.mod github dependencies, providing a simple way to say thanks to the maintainers of the modules you use and the contributors of Go itself.

GoThanks automatically stars Go's official repository and your go.mod github dependencies, providing a simple way  to say thanks to the maintainers of the modules you use and the contributors of Go itself.

Give thanks (in the form of a GitHub ★) to your fellow Go modules maintainers. About GoThanks performs the following operations Sends a star to Go's r

Dec 24, 2022
Takes a list of poker hands for two players in STDIN and scores them

Poker Hand Sorter Takes a list of poker hands for two players in STDIN and scores them. Returns the number of hands won by each player. Prerequisites

Jan 16, 2022
A command-line tool that triggers command when the input (doesn't) comes from STDIN in an interval.

conk A command-line tool that triggers command when the input (doesn't) comes from STDIN in an interval. Usage $ conk -h A command-line tool that trig

Apr 29, 2022
Testing the use of a golang wrapper around UserMode Linux for making stdin

This code is for testing the use of a golang wrapper around UserMode Linux for making stdin, stdout and stderr available to attach, detach and reattach to from the host using Unix sockets.

Dec 24, 2021
Filter out multiple strings from stdin.

minus The minus tool accepts data from stdin and filters out any arguments passed. It simplifies something like this: some_command | grep -vF thing1 |

Dec 20, 2021
Renda is a go library that repeatedly executes any processes

Renda is a go library that repeatedly executes any processes.

Feb 7, 2022
Like comm(1), but for any number of files

ncomm is like comm, but for any number of files, not just two files.

Sep 20, 2022
Numtow - Golang number to words converter

numtow golang library to convert number to words. Supported languages: kazakh, e

Dec 31, 2022
Allows verifying client's phone number

hone verification module for Elling - Elytrium Billing

Jun 4, 2022
a generic object pool for golang

Go Commons Pool The Go Commons Pool is a generic object pool for Golang, direct rewrite from Apache Commons Pool. Features Support custom PooledObject

Jan 5, 2023
Go worker pool

description This is a generic worker pool for the Go language. It's useful when you want to limit the number of goroutines running in parallel. instal

Jul 4, 2021