Flags-first package for configuration

ff go.dev reference Latest Release Build Status

ff stands for flags-first, and provides an opinionated way to populate a flag.FlagSet with configuration data from the environment. By default, it parses only from the command line, but you can enable parsing from environment variables (lower priority) and/or a configuration file (lowest priority).

Building a commandline application in the style of kubectl or docker? Consider package ffcli, a natural companion to, and extension of, package ff.

Usage

Define a flag.FlagSet in your func main.

import (
	"flag"
	"os"
	"time"

	"github.com/peterbourgon/ff/v3"
)

func main() {
	fs := flag.NewFlagSet("my-program", flag.ExitOnError)
	var (
		listenAddr = fs.String("listen-addr", "localhost:8080", "listen address")
		refresh    = fs.Duration("refresh", 15*time.Second, "refresh interval")
		debug      = fs.Bool("debug", false, "log debug information")
		_          = fs.String("config", "", "config file (optional)")
	)

Then, call ff.Parse instead of fs.Parse. Options are available to control parse behavior.

	ff.Parse(fs, os.Args[1:],
		ff.WithEnvVarPrefix("MY_PROGRAM"),
		ff.WithConfigFileFlag("config"),
		ff.WithConfigFileParser(ff.PlainParser),
	)

This example will parse flags from the commandline args, just like regular package flag, with the highest priority. (The flag's default value will be used only if the flag remains unset after parsing all provided sources of configuration.)

Additionally, the example will look in the environment for variables with a MY_PROGRAM prefix. Flag names are capitalized, and separator characters are converted to underscores. In this case, for example, MY_PROGRAM_LISTEN_ADDR would match to listen-addr.

Finally, if a -config file is specified, the example will try to parse it using the PlainParser, which expects files in this format.

listen-addr localhost:8080
refresh 30s
debug true

You could also use the JSONParser, which expects a JSON object.

{
	"listen-addr": "localhost:8080",
	"refresh": "30s",
	"debug": true
}

Or, you could write your own config file parser.

// ConfigFileParser interprets the config file represented by the reader
// and calls the set function for each parsed flag pair.
type ConfigFileParser func(r io.Reader, set func(name, value string) error) error

Flags and env vars

One common use case is to allow configuration from both flags and env vars.

package main

import (
	"flag"
	"fmt"
	"os"

	"github.com/peterbourgon/ff/v3"
)

func main() {
	fs := flag.NewFlagSet("myservice", flag.ExitOnError)
	var (
		port  = fs.Int("port", 8080, "listen port for server (also via PORT)")
		debug = fs.Bool("debug", false, "log debug information (also via DEBUG)")
	)
	ff.Parse(fs, os.Args[1:], ff.WithEnvVarNoPrefix())

	fmt.Printf("port %d, debug %v\n", *port, *debug)
}
$ env PORT=9090 myservice
port 9090, debug false
$ env PORT=9090 DEBUG=1 myservice -port=1234
port 1234, debug true
Owner
Peter Bourgon
The official GitHub account of Dwayne 'The Rock' Johnson
Peter Bourgon
Comments
  • toml dependency and modules

    toml dependency and modules

    This is a very nice package that I'm considering using.

    I don't want to introduce a toml dependency in my project. This is easily fixed as I will vendor this code and remove the toml-related code when I do. But it leaves me wondering: is there a better way to do this under modules? I thought about sending a PR with the toml code moved to a fftoml package, but that still leaves the module dependency. Is there some other way to do this?

  • ffcli: enhancements based on feedback

    ffcli: enhancements based on feedback

    • Add UsageFunc to allow precise control of -h output
    • Run → {Parse, Run} + ParseAndRun helper — breaking change
    • Wire context.Context thru Run and Exec (thanks @bobg)
    • Add motivation, goals, and non-goals to package docs
    • Add basic example to package docs
    • Add links to more complex examples to package docs: using io.Writer instead of os.Stdout, etc.
  • add .env file support

    add .env file support

    Hi,

    Given that I use Docker quite a lot when coding, I also use the .env file to pass configuration to the docker-compose. But given the speed of go run compared to docker-compose up -d --build, I want to test more with the former.

    So here is the package that helps me do that without sourcing the .env file in my shell (and mostly, forgetting about re-sourcing it).

    So now, running the app with a docker-compose up -d --build or go run does not require another workflow.

    It is a copy paste from the ff.PlainParser that use = as the key/value split. It also has an optional parser that accepts a prefix as the .env often has the variable with prefix to avoid conflicts with other services, as in env variables. I'm not sure it's the best way to do it, but I thought it was maybe the less intrusive since it's quite specific to the handling of .env files. I'm open to suggestions.

  • ffcli: Two-phase apps complicate noop handling

    ffcli: Two-phase apps complicate noop handling

    Noop commands (i.e., have subcommands but no exec) can be handled in a one-phase app pretty easily by having the command return flag.ErrHelp, so that users who accidentally call it get usage help instead of nothing. That’s good.

    But it gets awkward in a two-phase app, especially when the initialization involves logging output. A user would get usage help on stderr and logging output on stdout interleaved. Unless I’m missing something, this is not currently possible. It would be nice to be able to specify that Parse mimic the flag.ExitOnError functionality on noops, or at least a Noop() bool method so the initialization phase can be avoided.

    I’m happy to put together a PR for this, but I wanted to raise the issue first in case I’m missing something and to get your input on preferred solutions.

  • Add support for TOML tables to fftoml package

    Add support for TOML tables to fftoml package

    This adds support for TOML tables. It does so by concatenating nested keys with the - character.

    For example:

    [a]
    b = "c"
    

    Will set the flag -a-b with the value "c".

    var b string
    flagset.StringVar(&b, "a-b", "a string")
    // ...
    

    This swaps to a different toml parser (namely github.com/pelletier/go-toml), however, upon reflection the BurntSushi version is probably still useable. I can swap to attempt that if that would be more preferrable.

  • Only split environment variables on commas when explicitly told to do so?

    Only split environment variables on commas when explicitly told to do so?

    I ran into an issue yesterday when a string flag's value was being truncated, and I eventually figured out that it was ff splitting the environment variable on the comma, leaving the flag set to only the last few characters of the input.

    I believe that splitting by default is not a good idea because none of the default flag types actually support multiple values, meaning the default behaviour is effectively to (incorrectly) truncate input if there's a comma in it.

    Splitting is also currently an all-or-nothing proposition: you can have automatic splitting on if you've defined your own custom slice-based Value, but then you also run the risk of standard flag.String values being borked if their input contains a comma.

    I think a better solution would be for environment variable splitting to be explicitly opt-in for specific flags or specific types (if that's possible), e.g. ff.WithEnvVarSplit("flag1", "flag2", ...) or ff.WithEnvVarSplit(*CustomValue1, *CustomValue2, ...).

  • Update yaml.v2 and go-toml modules

    Update yaml.v2 and go-toml modules

    Generated with:

    go get -u ./...
    go mod tidy
    go test
    

    Builds fine and tests run fine as well. Runtime testing was limited to my own use case.

    This also brings yaml.v2 to a version which is not affected by CVE-2019-11254 anymore. For details also see: https://github.com/advisories/GHSA-wxc4-f4m6-wwqv

  • Feature request: Allow an extension for different sources

    Feature request: Allow an extension for different sources

    We have been happy users of this package since you created it and everything filled our needs up until now.

    Recently we added Vault to our toolchain and I tried to sketch what would be the simplest way to integrate it in our products.

    One of the ideas I come up with was to add an extension point to this package so that whenever it tries to solve a flag and if no other method was able to find it (flag, env, config), use this extension/resolver as a last resource.

    Would you be open to include such a feature?

  • Not sure how to properly to share config file between root flag and subcommands.

    Not sure how to properly to share config file between root flag and subcommands.

    I want something like this to work: mycmd -config configfile subcommand -subarg ... where one config file has values for both root and any subcommands.

    Having one config file per subcommand with it's own -config argument is too messy and user unfriendly to consider, the one root level -config should be enough and the config file should preferably not have to be reread multiple times.

    I'm not sure how to express this in a clean way using ffcli.

  • fix(ffcli): support sharing a flag with different default values

    fix(ffcli): support sharing a flag with different default values

    ~disclaimer: hi, this PR only contains a test case to reproduce a bug, but no fix yet~

    ~can someone help me understand how to fix it so I can finish the PR?~

    thanks

    edit: to support various edge cases, I added a Command.FlagSetBuilder optional callback

    fixes #68

  • Add Note for flag defaults when env var matches

    Add Note for flag defaults when env var matches

    Hi @peterbourgon - thanks so much for this library. I found it at just the right time to save me a bunch of work! One thing that I had to do some testing around to make sure I understood how it worked was how ff would handle flag defaults. I like the way it works currently, I just thought I could add a note in to help newcomers like myself understand how the logic around precedence works for flag defaults. Does this make sense? Thanks for considering! Dan

  • ffcli: mixing flags and positional arguments

    ffcli: mixing flags and positional arguments

    Is there a good strategy for doing something like this?

    In the example: https://github.com/peterbourgon/ff/blob/main/ffcli/examples/objectctl/pkg/createcmd/create.go allowing the --overwrite flag to come at the end of the line: objectctl create <key> <value data...> --overwrite. kubectl is a good example of a program that behaves this way. You can put cli flags damn near anywhere in it.

    I'm porting over a cli app from another language and would like to keep the same cli syntax that it has (and it has flags after the positional arg). But if you provide the flag at the end, it becomes part of the args parameter of Exec.

  • ffcli: use ShortHelp field as tagline

    ffcli: use ShortHelp field as tagline

    Hello @peterbourgon,

    Consider the help output of a program with subcommands:

    $ xprog -h
    USAGE
      xprog [flags] <command> ...
    
    SUBCOMMANDS
      ssh          upload and run the test binary via SSH
      passthrough  run the test binary directly on the host
    
    FLAGS
      -v=false  verbose output
    

    if then we ask the help for a subcommand we "loose" the ShortHelp:

    $ xprog ssh -h
    USAGE
      xprog ssh [flags] <test-binary> [go-test-flags]
    
    FLAGS
      -v=false  verbose output
    

    This proposal is to use the Name and ShortHelp field of the subcommand as a "tagline":

    $ xprog ssh -h
    ssh -- upload and run the test binary via SSH
    
    USAGE
      xprog ssh [flags] <test-binary> [go-test-flags]
    
    FLAGS
      -v=false  verbose output
    

    Actually this could be used also for the tagline of the root command itself:

    $ xprog -h
    xprog -- a test runner for "go test -exec"
    
    USAGE
      xprog [flags] <command> ...
    
    SUBCOMMANDS
    
    

    What do you think?

  • Feature request: multi-level flags

    Feature request: multi-level flags

    I often find myself repeating code like this:

    var verbose bool
    
    func init() {
    	all := []*flag.FlagSet{ /* list of FlagSets for different sub-commands and sub-sub-commands */ }
    	for _, fs := range all {
    		fs.BoolVar(&verbose, "v", false, "print debug information")
    	}
    }
    

    That way:

    $ cmd -v sub sub1
    $ cmd sub -v sub1
    $ cmd sub sub1 -v
    

    all do the same thing. (And it's not just -v. I find it frustrating as a user to have to figure out at which level a flag goes, so I generally want to be able to put them anywhere.)

    I'd like it if ff made this easier to accomplish.

    I'm not sure what the API would look like but it'd be nice if there was a way to say "these flags can be set by any subcommand of this command".

    Or maybe always do that and complain if there are any collisions? (That will also help avoid confusing UX in clients for which some flag means different things depending on where on the command line you add it.)

  • Provide more contextualized usage messages

    Provide more contextualized usage messages

    Usage messages should include the names and flags for all commands from the root to the terminal command. Previously usage functions could only access the terminal command's data, which made it difficult to automatically generate a fully contextuatlized usage message. The programmer was forced to manually copy the information into each subcommands usage function.

    This commit changes Command.Parse to collect the sequence of commands found from the beginning of the parse to the terminal command found. It also changes the signature of UsageFunc to accept the sequence of commands found during parsing. These changes provide implementations of UsageFunc with the data necessary to automatically generate more detailed usage messages with less work required by programmers.

    Note: The UsageFunc signature change is not backwards compatible and requires increasing the major version of the module.

  • ffcli: Default usage messages of subcommands lack information from preceding commands in the tree

    ffcli: Default usage messages of subcommands lack information from preceding commands in the tree

    Run the program below (or on the playground) to see an example of the problem.

    package main
    
    import (
    	"flag"
    
    	"github.com/peterbourgon/ff/v3/ffcli"
    )
    
    func main() {
    	barfs := flag.NewFlagSet("bar", flag.ContinueOnError)
    	barfs.String("bf", "", "bar flag")
    	bar := &ffcli.Command{
    		Name:      "bar",
    		FlagSet:   barfs,
    		ShortHelp: "bar help",
    	}
    
    	foofs := flag.NewFlagSet("foo", flag.ContinueOnError)
    	foofs.String("ff", "", "foo flag")
    	foo := &ffcli.Command{
    		Name:        "foo",
    		FlagSet:     foofs,
    		ShortHelp:   "foo help",
    		Subcommands: []*ffcli.Command{bar},
    	}
    
    	rootfs := flag.NewFlagSet("root", flag.ContinueOnError)
    	rootfs.String("rf", "", "root flag")
    	root := &ffcli.Command{
    		Name:        "root",
    		FlagSet:     rootfs,
    		ShortHelp:   "root help",
    		Subcommands: []*ffcli.Command{foo},
    	}
    
    	root.Parse([]string{"foo", "bar", "-h"})
    }
    

    It prints:

    USAGE
      bar
    
    FLAGS
      -bf ...  bar flag
    

    It should be obvious that the above help information is inadequate. The bar command is presented detached from its context as a subcommand of foo which is likewise a subcommand of root.

    I suggest that the help information should look more like this:

    USAGE
      root [flags] foo [flags] bar [flags]
    
    root FLAGS
      -rf ...  root flag
    
    foo FLAGS
      -ff ...  foo flag
    
    bar FLAGS
      -bf ...  bar flag
    

    We can write a custom UsageFunc to produce that help text, but that doesn't scale, is tedious to maintain, and couples subcommand metadata tightly to its position in the command tree. I suggest that ffcli should be able to create the above help text from only the metadata provided in the program shown above.

    I have prototyped a change that achieves that goal and will submit a PR showing that work for discussion. Note: The change requires a breaking API change.

  • Allow use of alternative FlagSet implementations

    Allow use of alternative FlagSet implementations

    Alternative FlagSet implementations such as https://github.com/spf13/pflag allow for different styles of flags.

    If ff and ffcli used a minimal interface with only the functions they call, in place of flag.FlagSet, this would allow users to use the FlagSet implementation of their choosing.

Related tags
persistent storage for flags in go

ingo is a simple Go library helping you to persist flags in a ini-like config file. Features and limitations Requires Go 1.5 or later automatically cr

Sep 26, 2022
Configure is a Go package that gives you easy configuration of your project through redundancy

Configure Configure is a Go package that gives you easy configuration of your project through redundancy. It has an API inspired by negroni and the fl

Sep 26, 2022
A golang package for parsing ini-style configuration files

Mini Mini is a simple ini configuration file parser. The ini syntax supported includes: The standard name=value Comments on new lines starting with #

Jan 7, 2023
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Dec 30, 2021
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Jan 8, 2023
12 factor configuration as a typesafe struct in as little as two function calls

Config Manage your application config as a typesafe struct in as little as two function calls. type MyConfig struct { DatabaseUrl string `config:"DAT

Dec 13, 2022
JSON or YAML configuration wrapper with convenient access methods.

Config Package config provides convenient access methods to configuration stored as JSON or YAML. This is a fork of the original version. This version

Dec 16, 2022
An opinionated configuration loading framework for Containerized and Cloud-Native applications.
An opinionated configuration loading framework for Containerized and Cloud-Native applications.

Opinionated configuration loading framework for Containerized and 12-Factor compliant applications. Read configurations from Environment Variables, an

Dec 16, 2022
Load configuration in cascade from multiple backends into a struct
Load configuration in cascade from multiple backends into a struct

Confita is a library that loads configuration from multiple backends and stores it in a struct. Supported backends Environment variables JSON files Ya

Jan 1, 2023
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Nov 3, 2022
A minimalist Go configuration library
A minimalist Go configuration library

fig fig is a tiny library for loading an application's config file and its environment into a Go struct. Individual fields can have default values def

Dec 23, 2022
go-up! A simple configuration library with recursive placeholders resolution and no magic.

go-up! A simple configuration library with placeholders resolution and no magic. go-up provides a simple way to configure an application from multiple

Nov 23, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
Go configuration made easy!

gofigure Go configuration made easy! Just define a struct and call Gofigure Supports strings, ints/uints/floats, slices and nested structs Supports en

Sep 26, 2022
Harvest configuration, watch and notify subscriber

Harvester Harvester is a configuration library which helps setting up and monitoring configuration values in order to dynamically reconfigure your app

Dec 26, 2022
go implementation of lightbend's HOCON configuration library https://github.com/lightbend/config

HOCON (Human-Optimized Config Object Notation) Configuration library for working with the Lightbend's HOCON format. HOCON is a human-friendly JSON sup

Dec 3, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022