Zero-downtime restarts in Go

goagain

Zero-downtime restarts in Go

The goagain package provides primitives for bringing zero-downtime restarts to Go applications that accept connections from a net.TCPListener or net.UnixListener.

Have a look at the examples because it isn't just a matter of importing the library and everything working. Your main function will have to accomodate the goagain protocols and your process will have to have some definition (however contrived you like) of a graceful shutdown process.

Installation

go get github.com/rcrowley/goagain

Usage

Send SIGUSR2 to a process using goagain and it will restart without downtime.

example/single/main.go: The Single strategy (named because it calls execve(2) once) operates similarly to Nginx and Unicorn. The parent forks a child, the child execs, and then the child kills the parent. This is easy to understand but doesn't play nicely with Upstart and similar direct-supervision init(8) daemons. It should play nicely with systemd.

example/double/main.go: The Double strategy (named because it calls execve(2) twice) is experimental so proceed with caution. The parent forks a child, the child execs, the child signals the parent, the parent execs, and finally the parent kills the child. This is regrettably much more complicated but plays nicely with Upstart and similar direct-supervision init(8) daemons.

Owner
Comments
  • Lack of examples of how to use it for use with http in the standard library

    Lack of examples of how to use it for use with http in the standard library

    I like the idea of goagain, but I'm surprised that so much code has to live in my main(). Can't we have a goagain.HttpListenAndserve()? This would be much more user friendly and solve the common case that one just wants to use go's builtin http server.

    Is there a good reason it is not like this?

  • fix another go get errors

    fix another go get errors

    go get github.com/rcrowley/goagain on ubuntu results in github.com/rcrowley/goagain/goagain.go:191: function ends without a return statement for me.

    this PR fixes this.

  • Add support for seamless re-execs without lost requests or requests sent to old processes

    Add support for seamless re-execs without lost requests or requests sent to old processes

    ... parent stops listening and lingers for ten seconds handling slow requests before giving up, while the child picks up the listening socket immediately and continues serving requests.

  • Leaks file descriptors on graceful reload

    Leaks file descriptors on graceful reload

    The first time it runs.

    goagain-e 9114 manuel 0u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9114 manuel 1u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9114 manuel 2u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9114 manuel 3u IPv4 678836 0t0 TCP 127.0.0.1:48879 (LISTEN) goagain-e 9114 manuel 4r FIFO 0,8 0t0 678837 pipe goagain-e 9114 manuel 5w FIFO 0,8 0t0 678837 pipe goagain-e 9114 manuel 6u 0000 0,9 0 6869 anon_inode goagain-e 9114 manuel 17r REG 0,3 0 410676 /proc/4972/auxv

    Every subsequent time, it adds two more sockets.

    goagain-e 9268 manuel 0u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9268 manuel 1u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9268 manuel 2u CHR 136,0 0t0 3 /dev/pts/0 goagain-e 9268 manuel 3u IPv4 678836 0t0 TCP 127.0.0.1:48879 (LISTEN) goagain-e 9268 manuel 4u IPv4 678836 0t0 TCP 127.0.0.1:48879 (LISTEN) goagain-e 9268 manuel 5r FIFO 0,8 0t0 688469 pipe goagain-e 9268 manuel 6w FIFO 0,8 0t0 688469 pipe goagain-e 9268 manuel 7u IPv4 678836 0t0 TCP 127.0.0.1:48879 (LISTEN) goagain-e 9268 manuel 8u 0000 0,9 0 6869 anon_inode goagain-e 9268 manuel 17r REG 0,3 0 410676 /proc/4972/auxv

    Where are you reopening the socket?

    Runtime go is the one that ships with Ubuntu 12.04.

  • Make stub functions for Windows (and possibly others)

    Make stub functions for Windows (and possibly others)

    Currently any project that includes goagain will be unbuildable on Windows due to syscall dependencies only present on some systems.

    Would it be possible to create stubs that will be used for unsupported systems via build tags?

    Of course there will not be any of the goagain functionality, but at least our developers on other platforms will be able to build projects using goagain.

    PS. Thanks for an awesome project!

  • How to build a service which supports the 'restart' operation with goagain?

    How to build a service which supports the 'restart' operation with goagain?

    Hi,

    Could you point me the direction how can I build the service which would support the "restart" operation?

    The big goal of mine, is to develop an application which I would be able to restart using the command like service apache2 restart This case, when I have new binary uploaded, the new entity would be the updated application (am I right?)

    As I understand the "goagain" is something I can use for this purpose. But I'm a bit lost about how should I use it.

    I run the "single" example - I tried to kill it, to interrupt it, but neither operation has lead to the fact that the application was restarted.

    1. How can I run the example app and see if it works?
    2. Do you have any idea what I should research to be able to add graceful restart / stop and start to my application?

    Thanks

  • goagain does not work!

    goagain does not work!

    Running the following httperf command... almost 4000 transactions are lost because it takes about a second to relaunch the app.

    httperf --num-calls=4000 --port= 48879 --uri=/mock --num-conns=10

    Looking closely at the code it's not possible to have a zero downtime.

    (a) goagain shares open connection (b) there is no state migration (c) during the swap no new connections can be made and therefore potential transactions lost

    /r

  • Doesn't finish ongoing requests on reload

    Doesn't finish ongoing requests on reload

    Unlike http://code.google.com/p/jra-go/source/browse/cmd/upgradable/main.go, the parent process doesn't actually stop listening or wait until it's done handling all requests, before it dies. This is because the child kills the parent and the parent exits unconditionally. This causes loss of requests.

  • Doesn't work in OSX

    Doesn't work in OSX

    I'm really confused why this doesn't work in OSX, the sub process always says that "dup: bad file descriptor" . But I have it working fine on linux. Any ideas of what to look at?

  • Go programs can handle SIGQUIT

    Go programs can handle SIGQUIT

    This tool does not handle SIGQUIT by default because it claims that Go "doesn't seem to like handling SIGQUIT". That may have been true in prior versions of Go, but in Go 1 SIGQUIT can be handled just like any other signal. Programs run under the "go run" tool aren't able to intercept SIGQUIT, but programs compiled to an executable and run work fine. Since a goal seems to be to mimic the behavior of Unicorn and nginx, I'd suggest that SIGQUIT can be used instead of SIGTERM.

  • Fails to compile:

    Fails to compile: "cannot use int(fd) (type int) as type syscall.Handle in function argument"

    Doing a go get github.com/rcrowley/goagain gives me:

    C:\gd\src\github.com\ezbiz>go get github.com/rcrowley/goagain
    # github.com/rcrowley/goagain
    c:\gd\src\github.com\rcrowley\goagain\goagain.go:25: undefined: syscall.SIGUSR2
    c:\gd\src\github.com\rcrowley\goagain\goagain.go:45: undefined: syscall.SIGUSR2
    c:\gd\src\github.com\rcrowley\goagain\goagain.go:71: cannot use int(fd) (type int) as type syscall.Handle in function argument
    c:\gd\src\github.com\rcrowley\goagain\goagain.go:92: undefined: syscall.Kill
    

    This is go version go1.0.3 64-bit under Windows 7 64-bit. It's strange because syscall.Kill and syscall.SIGUSR2 are certainly "defined" under Go 1.0.3 as per the official pkg-docs..

    Any idea what the issue could be here?

  • 
Fix function comments based on best practices from Effective Go

    Fix function comments based on best practices from Effective Go

    Every exported function in a program should have a doc comment. The first sentence should be a summary that starts with the name being declared. From effective go.

    PR generated by CodeLingo. Install here to drive Continuous Higher Standards.

  • single strategy error in mac

    single strategy error in mac

    In Single strategy๏ผŒsetEnvs error in mac.

    error message

    panic: reflect: call of reflect.Value.Int on zero Value
    
    goroutine 1 [running]:
    reflect.Value.Int(...)
    	/usr/local/go/src/reflect/value.go:961
    

    solution

    Replace with below code will solve this problems.

    v := reflect.ValueOf(l).Elem().FieldByName("fd").Elem().FieldByName("pfd")
    fd = uintptr(v.FieldByName("Sysfd").Int())
    
  • Drop resolving listener file descriptor by reflecting private struct fields.

    Drop resolving listener file descriptor by reflecting private struct fields.

    Drop resolving listener file descriptor by reflecting private struct fields. Instead we duplicate listener file descriptor, which handles closeOnExec syscalls.

    The current implementation uses reflect package, and is prone to break (actually as of go 1.9 it WILL break, because the netFD struct fields have been moved around and sysfd field does not exist anymore, and is reachable instead by pfd poll.FD field).

  • Fix double-close of FD on garbage collection (#25)

    Fix double-close of FD on garbage collection (#25)

    Fixes a rogue close() that happens whenever the GC next runs after Listen(). NewFile() takes responsibility for the fd it is passed but Listen() then proceeds to manually syscall.Close() the fd anyway. So when the file object gets finalized it is closed a second time, either causing an EBADF or closing some other random file.

  • Allow logging to be overridden

    Allow logging to be overridden

    I agree with #2 and #3 that logging should not forced upon users of this library. But I also agree with @rcrowley that the logging is useful and needed. This PR doesn't change the default logging functionality but adds a Logger var on the package that can be set to override where log messages go.

    This can be useful for applications that already have a logger setup and want the goagain logs to go to the same place. You just set

    myLogger := log.New(myWriter, "err: ", log.LstdFlags)
    goagain.Logger = myLogger
    

    Or if you wanted to disable logging altogether, you can set

    goagain.Logger = nil
    
Related tags
Graceful process restarts in Go

Graceful process restarts in Go It is sometimes useful to update the running code and / or configuration of a network service, without disrupting exis

Dec 27, 2022
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Jan 5, 2023
llb - It's a very simple but quick backend for proxy servers. Can be useful for fast redirection to predefined domain with zero memory allocation and fast response.

llb What the f--k it is? It's a very simple but quick backend for proxy servers. You can setup redirect to your main domain or just show HTTP/1.1 404

Sep 27, 2022
Simple web content/proxy server that embodies enterprise zero trust security

pswa - Protected Static Web App Introduction pswa is a simple web content/proxy server which is suitable for various static web apps. Features Availab

Nov 11, 2022
Fake server, Consumer Driven Contracts and help with testing performance from one configuration file with zero system dependencies and no coding whatsoever
Fake server, Consumer Driven Contracts and help with testing performance from one configuration file with zero system dependencies and no coding whatsoever

mockingjay server Mockingjay lets you define the contract between a consumer and producer and with just a configuration file you get: A fast to launch

Jan 6, 2023
Control your Flipper Zero over Protobuf RPC protocol.

go-flipper Control your Flipper Zero over Protobuf RPC protocol. This library is designed to be transport agnostic, though I've tested it with RPC ove

Dec 17, 2022
Raspberry Pi Zero W IR remote webserver for Cambridge Audio CXA81 Amplifier

CXA81-IR-Remote-Server About The Project I initially wanted to control my Cambri

Dec 27, 2021
โ€œDear Port80โ€ is a zero-config TCP proxy server that hides SSH connection behind a HTTP server!

Dear Port80 About The Project: โ€œDear Port80โ€ is a zero-config TCP proxy server that hides SSH connection behind a HTTP server! +---------------------

Jun 29, 2022
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

Nov 1, 2022
Zero-downtime restarts in Go

goagain Zero-downtime restarts in Go The goagain package provides primitives for bringing zero-downtime restarts to Go applications that accept connec

Dec 28, 2022
Builds and restarts a Go project when it crashes or some watched file changes
Builds and restarts a Go project when it crashes or some watched file changes

gaper Used to build and restart a Go project when it crashes or some watched file changes Aimed to be used in development only. Changelog See Releases

Dec 30, 2022
Graceful process restarts in Go

Graceful process restarts in Go It is sometimes useful to update the running code and / or configuration of a network service, without disrupting exis

Dec 27, 2022
A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.
A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.

A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.

Feb 6, 2022
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Jan 2, 2023
Zero allocation Nullable structures in one library with handy conversion functions, marshallers and unmarshallers

nan - No Allocations Nevermore Package nan - Zero allocation Nullable structures in one library with handy conversion functions, marshallers and unmar

Dec 20, 2022
Zero boilerplate database operations for Go
Zero boilerplate database operations for Go

(Now compatible with MySQL and PostgreSQL!) Everyone knows that performing simple DATABASE queries in Go takes numerous lines of code that is often re

Dec 17, 2022
BPG decoder for Go (Zero Dependencies).

Go่ฏญ่จ€QQ็พค: 102319854, 1055927514 ๅ‡น่ฏญ่จ€(ๅ‡น่ฏป้Ÿณโ€œWaโ€)(The Wa Programming Language): https://github.com/wa-lang/wa BPG for Go BPG is defined at: http://bellard.o

Sep 7, 2020