Process manager for Procfile-based applications

Hivemind

Build Status

Hivemind is a process manager for Procfile-based applications. At the moment, it supports Linux, FreeBSD, and macOS.

Procfile is a simple format to specify types of processes your application provides (such as web application server, background queue process, front-end builder) and commands to run those processes. It can significantly simplify process management for developers and is used by popular Platforms-as-a-Service, such as Heroku and Deis. You can learn more about the Procfile format here.

There are some good Procfile-based process management tools, including foreman by David Dollar, which started it all. The problem with most of those tools is that processes you want to manage start to think they are logging their output into a file, and that can lead to all sorts of problems: severe lagging, losing or breaking colored output. Tools can also add vanity information (unneeded timestamps in logs). Hivemind was created to fix those problems once and for all.

See this article for a good intro and all the juicy details! Introducing Overmind and Hivemind

Sponsored by Evil Martians

Enter Hivemind

Hivemind uses pty to capture process output. That fixes any problem with log clipping, delays, and TTY colors other process management tools may have.

If you would like a process management tool with a lot of features, including tmux support, restarting and killing individual processes and advanced configuration, you should take a look at Hivemind's big brother — Overmind!

Installation

With Homebrew (macOS)

brew install hivemind

Download the latest Hivemind release binary

You can download the latest release here.

From Source

You need Go 1.11 or later to build the project.

$ GO111MODULE=on go get -u -f github.com/DarthSim/hivemind

Note: You can update Hivemind the same way.

Usage

Hivemind works with a Procfile. It may look like this:

web: bin/rails server
worker: bundle exec sidekiq
assets: gulp watch

To get started, you just need to run Hivemind from your working directory containing Procfile.

$ hivemind

If Procfile isn't located in your working directory, or named it non-standard as Procfile.dev, you can specify the path to it: [Fun Fact: Name of the Procfile is arbitrary and can be anything, although it is a best practice to name it as Procfile for sanity]

$ hivemind path/to/your/Procfile
$ hivemind path/to/your/Procfile.dev

Run hivemind --help to see other options. Note that every Hivemind option can be set with corresponding environment variable.

Environment

If you need to set specific environment variables before running a Procfile, you can specify them in the .env file in the current working directory. The file should contain variable=value pairs, one per line:

PATH=$PATH:/additional/path
PORT=3000
HIVEMIND_TITLE=my_awsome_app

Author

Sergey "DarthSim" Aleksandrovich

Highly inspired by Foreman.

Many thanks to @antiflasher for the awesome logo.

License

Hivemind is licensed under the MIT license.

See LICENSE for the full license text.

Owner
Sergey Alexandrovich
Kendo-mouse from Mars
Sergey Alexandrovich
Comments
  • Installation via `go get` as documented in README fails

    Installation via `go get` as documented in README fails

    With go 1.13 on Centos 7

    [vagrant@localhost ~]$ go version
    go version go1.13.6 linux/amd64
    [vagrant@localhost ~]$ go get -u -f github.com/DarthSim/hivemind
    # github.com/DarthSim/hivemind
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:26:5: app.Author undefined (type *cli.App has no field or method Author)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:27:5: app.Email undefined (type *cli.App has no field or method Email)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:33:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in array or slice literal:
            cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:33:36: unknown field 'EnvVar' in struct literal of type cli.StringFlag
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:34:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in array or slice literal:
            cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:34:40: unknown field 'EnvVar' in struct literal of type cli.StringFlag
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:35:14: cannot use cli.IntFlag literal (type cli.IntFlag) as type cli.Flag in array or slice literal:
            cli.IntFlag does not implement cli.Flag (Apply method has pointer receiver)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:35:32: unknown field 'EnvVar' in struct literal of type cli.IntFlag
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:36:14: cannot use cli.IntFlag literal (type cli.IntFlag) as type cli.Flag in array or slice literal:
            cli.IntFlag does not implement cli.Flag (Apply method has pointer receiver)
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:36:37: unknown field 'EnvVar' in struct literal of type cli.IntFlag
    /opt/gopath/src/github.com/DarthSim/hivemind/main.go:36:14: too many errors
    [vagrant@localhost ~]$ ( cd /opt/gopath/src/github.com/DarthSim/hivemind/ && git status -v && git log | head -n5 )
    # On branch master
    nothing to commit, working directory clean
    commit 10d1fa0ff72d08fd1110f2c17a5bc0b1714af4af
    Author: DarthSim <[email protected]>
    Date:   Wed Dec 18 16:59:01 2019 +0600
    
        Update readme
    

    With go 1.12 on osx

    is-mbp-timothy4:tmp timothy$ go version
    go version go1.12.1 darwin/amd64
    is-mbp-timothy4:tmp timothy$ go get -u -f github.com/DarthSim/hivemind
    # github.com/DarthSim/hivemind
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:26:5: app.Author undefined (type *cli.App has no field or method Author)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:27:5: app.Email undefined (type *cli.App has no field or method Email)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:33:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in array or slice literal:
    	cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:33:36: unknown field 'EnvVar' in struct literal of type cli.StringFlag
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:34:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in array or slice literal:
    	cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:34:40: unknown field 'EnvVar' in struct literal of type cli.StringFlag
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:35:14: cannot use cli.IntFlag literal (type cli.IntFlag) as type cli.Flag in array or slice literal:
    	cli.IntFlag does not implement cli.Flag (Apply method has pointer receiver)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:35:32: unknown field 'EnvVar' in struct literal of type cli.IntFlag
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:36:14: cannot use cli.IntFlag literal (type cli.IntFlag) as type cli.Flag in array or slice literal:
    	cli.IntFlag does not implement cli.Flag (Apply method has pointer receiver)
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:36:37: unknown field 'EnvVar' in struct literal of type cli.IntFlag
    /Users/timothy/go/src/github.com/DarthSim/hivemind/main.go:36:14: too many errors
    is-mbp-timothy4:tmp timothy$ ( cd /Users/timothy/go/src/github.com/DarthSim/hivemind/ && git status -v && git log | head -n5 )
    On branch master
    Your branch is up to date with 'origin/master'.
    
    nothing to commit, working tree clean
    commit 10d1fa0ff72d08fd1110f2c17a5bc0b1714af4af
    Author: DarthSim <[email protected]>
    Date:   Wed Dec 18 16:59:01 2019 +0600
    
        Update readme
    
  • how to install binary on linux?

    how to install binary on linux?

    given i want to install a linux binary, what steps do i take? i'm on debian arm64. and would like to grab the binary so it's easy for me to use within github's codespaces. apologies, feeling like an absolute noob here!

    $ dpkg --print-architecture
    amd64
    $ curl -OL "https://github.com/DarthSim/hivemind/releases/download/v1.1.0/hivemind-v1.1.0-linux-arm64.gz"
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100   667  100   667    0     0   4446      0 --:--:-- --:--:-- --:--:--  4446
    100 1105k  100 1105k    0     0  3734k      0 --:--:-- --:--:-- --:--:-- 3734k
    $ gunzip hivemind-v1.1.0-linux-arm64.gz 
    $ chmod +x ./hivemind-v1.1.0-linux-arm64 
    $ ./hivemind-v1.1.0-linux-arm64 
    bash: ./hivemind-v1.1.0-linux-arm64: cannot execute binary file: Exec format error
    
  • Error installing Hivemind with Go 1.13

    Error installing Hivemind with Go 1.13

    Running go get github.com/DarthSim/[email protected] I got the following error:

    build github.com/DarthSim/hivemind: cannot load gopkg.in/urfave/cli.v1: cannot find module providing package gopkg.in/urfave/cli.v1
    

    I found a similar bug report and worked around it with replace gopkg.in/urfave/cli.v1 => github.com/urfave/cli v1.19.1 (the same version used in hivemind's go.mod).

    https://github.com/DarthSim/hivemind/blob/f67556693e716db78ad03c034bdd340eaad0645c/go.mod#L9

  • Fault creating process

    Fault creating process

    Just upgraded to the latest and running within a Docker container built from either the official Node 8.12 image or the official Ruby 2.6 image, after installing Go, Hivemind faults on startup:

    panic: runtime error: invalid memory address or nil pointer dereference
    react_1  | [signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x402fee]
    react_1  |
    react_1  | goroutine 1 [running]:
    react_1  | panic(0x565100, 0xc420010050)
    react_1  | 	/usr/lib/go-1.7/src/runtime/panic.go:500 +0x1a1
    react_1  | main.newProcess(0xc420010750, 0x3, 0xc420010755, 0xb, 0x2, 0xc4200105c0, 0x6, 0x1388, 0xc42000a800, 0xc4200105c0)
    react_1  | 	/go/src/github.com/DarthSim/hivemind/process.go:32 +0x34e
    react_1  | main.newHivemind(0x0, 0x0, 0x7ffedfba6edb, 0xc, 0x0, 0x0, 0xc4200105c0, 0x6, 0x1388, 0x64, ...)
    react_1  | 	/go/src/github.com/DarthSim/hivemind/hivemind.go:52 +0x335
    react_1  | main.main.func1(0xc42009c000, 0xc42009c000, 0xc420035a87)
    react_1  | 	/go/src/github.com/DarthSim/hivemind/main.go:67 +0x16d
    react_1  | github.com/DarthSim/hivemind/vendor/gopkg.in/urfave/cli%2ev1.HandleAction(0x55e720, 0xc42000a1a0, 0xc42009c000, 0x0, 0x0)
    react_1  | 	/go/src/github.com/DarthSim/hivemind/vendor/gopkg.in/urfave/cli.v1/app.go:485 +0xd4
    react_1  | github.com/DarthSim/hivemind/vendor/gopkg.in/urfave/cli%2ev1.(*App).Run(0xc4200981a0, 0xc42000a140, 0x2, 0x2, 0x0, 0x0)
    react_1  | 	/go/src/github.com/DarthSim/hivemind/vendor/gopkg.in/urfave/cli.v1/app.go:259 +0x74f
    react_1  | main.main()
    react_1  | 	/go/src/github.com/DarthSim/hivemind/main.go:72 +0x6e9
    

    All was good until the latest change.

  • use PORT= in .env

    use PORT= in .env

    It appears that hivemind 1.0.4 always sets its own value for PORT, even if PORT=1234 is present in .env.

    Would it be possible for hivemind to set PORT only if PORT is not already set? Changing this would also make this compatible with foreman's behavior on this particular matter (that is, it allows PORT to be configured via .env).

  • hivemind takes 99% CPU

    hivemind takes 99% CPU

    Hey, I noticed that hivemind takes 99% CPU all the time. Initially, I thought it's something w/ my processes, but nope.

    Procfile

    fake: sleep 10000
    

    results in

    screen shot 2017-11-09 at 12 00 48

    Same when I run my app.

    macOS: 10.12.6 Processor: 2,9 GHz Intel Core i7 Memory: 16 GB 2133 MHz LPDDR3

  • Ignore release command?

    Ignore release command?

    Hi,

    thanks for hivemind, I really enjoy using it!

    I've recently added a release command to my Procfile (see https://devcenter.heroku.com/articles/release-phase). And now hivemind stops working as it recognizes the task exit and shuts everything down.

    Would it make sense to ignore release by default? That's the behavior of heroku local

    Cheers, Florian

  • Add vendoring workflow to README.md

    Add vendoring workflow to README.md

    Currently there's no vendoring workflow noted in the README.md

    At a best guess the gvt tool seems compatible, but gvm is mentioned in the commit logs.

    Could this be clarified?

  • Add support for TTY detection and line-delimited JSON output.

    Add support for TTY detection and line-delimited JSON output.

    Two small-ish patches which add functionality useful when using Hivemind as a docker container init.

    The first adds TTY detection to disable color codes when not writing to a TTY output - making Hivemind compatible with line-based log collection systems or when run as a daemon.

    The second extends the first by adding line-delimited JSON output functionality for easy integration with ELK or other systems which can parse JSON or nested JSON.

    Due to the lack of a clearly good Golang ascii-escape library, the color coding is implemented in a rather brute-force way.

  • hivemind hangs when handling more than 65536 bytes of output

    hivemind hangs when handling more than 65536 bytes of output

    Hivemind hangs when it gets more than 65536 bytes of output from the child process. The following procfile demonstrates the problem:

    works: hexdump -v -n 32768 -e '1/1 "%02x"' /dev/urandom
    hangs: hexdump -v -n 32769 -e '1/1 "%02x"' /dev/urandom
    

    I'm on MacOS 10.15 Catalina.

  • Capability to run without a shell.

    Capability to run without a shell.

    Appears to work fine if not running under a shell. Useful for clean, from scratch containers.

    Ignore if there's a reason the command needs to run under a shell. Regards.

  • hivemind fails to start some processes properly, but foreman works as expected

    hivemind fails to start some processes properly, but foreman works as expected

    Summary

    The following Procfile works properly with foreman 0.87.2, but fails to execute both processes when called with hivemind 1.1.0 as installed via Homebrew. Go reports itself as go version go1.18.1 darwin/amd64.

    Procfile

    yard: open -a Safari.app http://localhost: 8088
    coverage: open -a Safari.app coverage/index.html
    

    Failures from Hivemind

    With foreman start, a PID is created for each browser process, and the relevant pages are opened in Safari. However, with hivemind ./Procfile the first process starts correctly even though hivemind thinks it has immediately exited. The second process (coverage) never runs, and reports:

    coverage: Running
    yard: Running...
    yard: Process exited
    coverage: Interrupting...
    coverage: Signal: interrupt
    

    This intuitively seems to be an issue with jobs that fork or share a parent process, since in actuality the YARD server (started separately by Guard in this case) continues to run, but hivemind seems to think that because open has exited it should not start or continue the other defined processes defined in the Procfile.

    If there's a work-around, or if I've misunderstood the documentation, please let me know. Otherwise, this seems like a bug for anything that might spawn background processes or have non-interdependent processes that should not fail to start or continue to run simply because another named process within the Procfile did.

  • Option to skip reading .env?

    Option to skip reading .env?

    Hi folks,

    Would it be possible to add a flag that skips reading a local .env file? It seems to get loaded no matter what, which isn't always desired.

    It would also be good to avoid setting things like $PORT too

    Example:

    Procfile

    test: sh -c 'echo FOO is $FOO'
    

    .env

    FOO=BAR
    
    ❯ hivemind -l test
    test | Running...
    test | FOO is bar
    test | Process exited
    
  • Hivemind process exits 0 regardless of child process status

    Hivemind process exits 0 regardless of child process status

    To my understanding, the parent process should exit with the same status as the child process it's running (if there's only one child process maybe), or some other non-zero status.

    Example:

    Procfile:

    test: exit 1
    
    ❯ hivemind -l test
    test | Running...
    test | exit status 1
    
    ❯ echo $?
    0
    

    Thanks in advance!

  • Remove godotenv (which breaks dotenv) and build a macos arm version

    Remove godotenv (which breaks dotenv) and build a macos arm version

    @DarthSim -

    a couple things here, and I didn't mess with tests, so very much assume this is not mergeable as is.... however.

    the big thing is removing the godotenv module. the reason being, dotenv allows you to overload -- so, you can define project envs in .env, and say local overrides in .env.local, for example. a nice way to prevent committing secrets.

    for safety, however, ruby's dotenv won't override existing env vars, typically those defined outside the scope of its runtime.

    godotenv doesn't understand this functionality, so it's just loading .env and nothing else -- and because those vars are defined before ruby starts, ruby's dotenv won't override them, and so local env settings are busted.

    tbh, i don't think it's appropriate for hivemind to be including them anyhow. if people want them and they're not already available, it's pretty easy to run a shell/pre-command to define the env within the Procfile... whereas being forced to define them via godotenv breaks things. hence the PR. :)

    also, i added a line to build a macos arm64 version, because m1.

    hope it helps!

  • Timeout when committing large files via gRPC in background jobs

    Timeout when committing large files via gRPC in background jobs

    We're looking at replacing foreman with Hivemind at GitHub and we're running into timeouts when trying to commit large files via gRPC in background jobs. Once we hit the timeout, Hivemind becomes unresponsive and we have to manually kill it. The jobs are able to commit small files so it seems like something about the file size is causing the timeout. Maybe there's a config setting or something similar we can change that will let the commit finish for large files?

    We haven't had this issue with either foreman or Overmind.

Manage Procfile-based applications

Foreman Manage Procfile-based applications Installation $ gem install foreman Ruby users should take care not to install foreman in their project's G

Dec 30, 2022
Weave Ignite is an open source Virtual Machine (VM) manager with a container UX and built-in GitOps management.
Weave Ignite is an open source Virtual Machine (VM) manager with a container UX and built-in GitOps management.

Weave Ignite is an open source Virtual Machine (VM) manager with a container UX and built-in GitOps management.

Nov 16, 2021
This vitual os application consist of 3 mini applications embedded in it like weather app , text editor and calculator .

Virtual-Operating-System This vitual os application consist of 3 mini applications embedded in it like weather app , text editor and calculator . APPS

Nov 11, 2021
A bytecode-based virtual machine to implement scripting/filtering support in your golang project.

eval-filter Implementation Scripting Facilities Types Built-In Functions Conditionals Loops Functions Case/Switch Use Cases Security Denial of service

Dec 30, 2022
Create virtual machines and run Linux-based operating systems in Go using Apple Virtualization.framework.

vz - Go binding with Apple Virtualization.framework vz provides the power of the Apple Virtualization.framework in Go.

Jan 9, 2023
Expr – a tiny stack-based virtual machine written in Go

Expr – a tiny stack-based virtual machine written in Go The executor is designed to interpret a simple expression language and it's useful in delegati

Nov 11, 2022
An interpreter written in go for a brainfuck-based language called €*

eurostar-go-interpreter This is an interpreter written in go for a brainfuck-bas

Sep 14, 2022
Process manager for Procfile-based applications and tmux
Process manager for Procfile-based applications and tmux

Overmind Overmind is a process manager for Procfile-based applications and tmux. With Overmind, you can easily run several processes from your Procfil

Jan 4, 2023
Manage Procfile-based applications

Foreman Manage Procfile-based applications Installation $ gem install foreman Ruby users should take care not to install foreman in their project's G

Dec 30, 2022
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Nov 10, 2022
Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Apr 21, 2022
High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

[RR2-BETA] RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports runnin

Jan 4, 2023
High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Jan 1, 2023
High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Dec 30, 2022
exo: a process manager & log viewer for dev
 exo: a process manager & log viewer for dev

exo: a process manager & log viewer for dev exo- prefix – external; from outside. Features Procfile compatible process manager.

Dec 28, 2022
Composer is a simple process manager for dev environments.

Composer Composer is a simple service manager for dev environments. How to build/install it? To build composer under ./bin, run: make build To build

May 12, 2022
High-performance PHP application server, load-balancer and process manager written in Golang
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Dec 9, 2021
RoadRunner: an open-source high-performance PHP application server, load balancer, and process manager
RoadRunner: an open-source high-performance PHP application server, load balancer, and process manager

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Jan 13, 2022
Rustpm - Process manager and automated updates for RustDedicated
Rustpm - Process manager and automated updates for RustDedicated

rustpm (WIP) Process manager for RustDedicated A drop in replacement for RustDed

Feb 15, 2022
May 11, 2023