Load configuration from multiple sources in Go

Multiconfig GoDoc Build Status

Load configuration from multiple sources. Multiconfig makes loading/parsing from different configuration sources an easy task. The problem with any app is that with time there are many options how to populate a set of configs. Multiconfig makes it easy by dynamically creating all necessary options. Checkout the example below to see it in action.

Features

Multiconfig is able to read configuration automatically based on the given struct's field names from the following sources:

  • Struct tags
  • TOML file
  • JSON file
  • YAML file
  • Environment variables
  • Flags

Install

go get github.com/koding/multiconfig

Usage and Examples

Lets define and struct that defines our configuration

type Server struct {
	Name    string `required:"true"`
	Port    int    `default:"6060"`
	Enabled bool
	Users   []string
}

Load the configuration into multiconfig:

// Create a new DefaultLoader without or with an initial config file
m := multiconfig.New()
m := multiconfig.NewWithPath("config.toml") // supports TOML, JSON and YAML

// Get an empty struct for your configuration
serverConf := new(Server)

// Populated the serverConf struct
err := m.Load(serverConf) // Check for error
m.MustLoad(serverConf)    // Panic's if there is any error

// Access now populated fields
serverConf.Port // by default 6060
serverConf.Name // "koding"

Run your app:

# Sets default values first which are defined in each field tag value. 
# Starts to load from config.toml
$ app

# Override any config easily with environment variables, environment variables
# are automatically generated in the form of STRUCTNAME_FIELDNAME
$ SERVER_PORT=4000 SERVER_NAME="koding" app

# Or pass via flag. Flags are also automatically generated based on the field
# name
$ app -port 4000 -users "gopher,koding"

# Print dynamically generated flags and environment variables:
$ app -help
Usage of app:
  -enabled=true: Change value of Enabled.
  -name=Koding: Change value of Name.
  -port=6060: Change value of Port.
  -users=[ankara istanbul]: Change value of Users.

Generated environment variables:
   SERVER_NAME
   SERVER_PORT
   SERVER_ENABLED
   SERVER_USERS

License

The MIT License (MIT) - see LICENSE for more details

Comments
  • -help panicking with Go 1.7

    -help panicking with Go 1.7

    After upgrading to Go 1.7 (and 1.7.1), -help gives a panic:

    $ brew switch go 1.6.3
    Cleaning /usr/local/Cellar/go/1.6.3
    Cleaning /usr/local/Cellar/go/1.7
    Cleaning /usr/local/Cellar/go/1.7.1
    3 links created for /usr/local/Cellar/go/1.6.3
    $ go build
    $ ./gosonic -help
    Usage of ./gosonic:
      -dbpath
            Change value of DbPath. (default ./devDb)
      -disabledownsampling
            Change value of DisableDownsampling.
      -disablevalidation
            Change value of DisableValidation. (default true)
    ...
    
    Generated environment variables:
       GOSONIC_PORT
       GOSONIC_MUSICFOLDER
       GOSONIC_DBPATH
    ...
    
    $ brew switch go 1.7.1
    Cleaning /usr/local/Cellar/go/1.6.3
    Cleaning /usr/local/Cellar/go/1.7
    Cleaning /usr/local/Cellar/go/1.7.1
    3 links created for /usr/local/Cellar/go/1.7.1
    $ go build
    $ ./gosonic -help
    Usage of ./gosonic:
    panic: reflect: call of reflect.Value.Interface on zero Value
    
    goroutine 1 [running]:
    panic(0x5c8280, 0xc4202c0360)
        /usr/local/Cellar/go/1.7.1/libexec/src/runtime/panic.go:500 +0x1a1
    reflect.valueInterface(0x0, 0x0, 0x0, 0x28744d88401, 0x0, 0xd6e84)
        /usr/local/Cellar/go/1.7.1/libexec/src/reflect/value.go:912 +0x204
    reflect.Value.Interface(0x0, 0x0, 0x0, 0x11bcb, 0x6089a0)
        /usr/local/Cellar/go/1.7.1/libexec/src/reflect/value.go:907 +0x44
    github.com/deluan/gosonic/vendor/github.com/fatih/structs.(*Field).Value(0xc4202d0360, 0xf547, 0x5dc700)
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/vendor/github.com/fatih/structs/field.go:31 +0x40
    github.com/deluan/gosonic/vendor/github.com/koding/multiconfig.(*fieldValue).String(0xc4202d0360, 0x5ef2a0, 0xc4202d0360)
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/vendor/github.com/koding/multiconfig/flag.go:151 +0x2f
    flag.isZeroValue(0xc4202ab180, 0xc420297f00, 0x7, 0xc4202c0180)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:393 +0x12d
    flag.(*FlagSet).PrintDefaults.func1(0xc4202ab180)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:467 +0x1d5
    flag.(*FlagSet).VisitAll(0xc4202bcea0, 0xc420053ab8)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:323 +0x67
    flag.(*FlagSet).PrintDefaults(0xc4202bcea0)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:476 +0x46
    github.com/deluan/gosonic/vendor/github.com/koding/multiconfig.(*FlagLoader).Load.func1()
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/vendor/github.com/koding/multiconfig/flag.go:67 +0x118
    flag.(*FlagSet).usage(0xc4202bcea0)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:828 +0x5c
    flag.(*FlagSet).parseOne(0xc4202bcea0, 0x617be0, 0x1, 0xc4202d40f0)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:870 +0xb4c
    flag.(*FlagSet).Parse(0xc4202bcea0, 0xc420094070, 0x1, 0x1, 0x0, 0x0)
        /usr/local/Cellar/go/1.7.1/libexec/src/flag/flag.go:915 +0x60
    github.com/deluan/gosonic/vendor/github.com/koding/multiconfig.(*FlagLoader).Load(0xc4202bb900, 0x584160, 0xc420121810, 0xc420139ee0, 0x0)
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/vendor/github.com/koding/multiconfig/flag.go:82 +0x25f
    github.com/deluan/gosonic/conf.LoadFromFlags()
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/conf/configuration.go:33 +0x79
    main.main()
        /Users/deluan/Development/go/src/github.com/deluan/gosonic/main.go:15 +0x2b
    

    For reference, my code is available here: https://github.com/deluan/gosonic/blob/master/conf/configuration.go

  • When working with glog,It gets flag provided but not defined: -alsologtostderr

    When working with glog,It gets flag provided but not defined: -alsologtostderr

    Because of the flagSet.Parse(args) ignored the flags defined by glog,I added flag.VisitAll before it.

    Works fine.

    
        flag.VisitAll(func(ff *flag.Flag) {
            f.flagSet.Var(ff.Value, ff.Name, ff.Usage)
        })
    
  • How could I use CamelCase?

    How could I use CamelCase?

    It is hard to use,

        m := multiconfig.New()
        m.MustLoad(&config)
        l := multiconfig.EnvironmentLoader{
            CamelCase: true,
        }
        l.Load(&config)
    
  • Using

    Using "go test" flags causes multiconfig to exit with a non-zero status

    The issue appears to be caused by "go test" passing its flags (with some reformatting, like -v turning into -test.v) to the test code and multiconfig is concerned because it is unaware of those flags.

    I've pushed a branch with code that demonstrates the issue: https://github.com/four2five/gotest_multiconfig_example

  • Add CamelCase support to TomlReader

    Add CamelCase support to TomlReader

    Would be nice for consistency.

    PS. Is this still maintained? We really like it but there's a few niggles here and there we'd like to change, like adding a name tag to make this kind of camelcase / snake_case be more easily configured via the config struct.

  • add full support for time.Duration

    add full support for time.Duration

    Support for time.Duration was added in #28, but its usage is inconsistent between the different loaders. While it is possible to specify a duration as 10s as an ENV variable, the same is not possible e.g. for TOML files. Here, you'd have to set the same duration as 10000000000.

  • Accessing non-option arguments

    Accessing non-option arguments

    How do I access non-option command line arguments?

    For example, running ./app -opt foo bar baz won't complain about bar and baz, but I also don't see any methods in multiconfig giving me access to these. flag.Args() doesn't work (acts as if flag hasn't been used); os.Args (expectedly) yields the entire command line.

  • Part of #23 - allow flagloader to set optional function to set flag usage

    Part of #23 - allow flagloader to set optional function to set flag usage

    @fatih This allow people to have function to set flag default Usage messages. To write a test I had to save the temporary FlagSet that is created (as a private field in FlagLoader). If you don't like this, we can either just delete the test (not great) or figure out another way to emit the usage and capture stdout/stderr (really gross).

    merge, comment or suggestions all welcome!

    n

  • panic: not struct

    panic: not struct

    @cihangir commit 15a9baf17e5dc67025d30dd65c98bb7e2edc3dd1 seems to have introduced a bug.

    panic: not struct
    
    goroutine 1 [running]:
    panic(0x8ff0a0, 0xc820202b90)
        /usr/lib/go/src/runtime/panic.go:481 +0x3e6
    github.com/fatih/structs.strctVal(0x8d8520, 0xc8201cb3b0, 0x0, 0x0, 0x0)
        /home/jonaz/go/src/github.com/fatih/structs/structs.go:439 +0x138
    github.com/fatih/structs.(*Field).FieldOk(0xc8201dc6c0, 0xc82020295c, 0x4, 0x3, 0xa0a840)
        /home/jonaz/go/src/github.com/fatih/structs/field.go:129 +0x107
    github.com/fatih/structs.(*Field).Field(0xc8201dc6c0, 0xc82020295c, 0x4, 0xad1120)
        /home/jonaz/go/src/github.com/fatih/structs/field.go:109 +0x39
    github.com/koding/multiconfig.(*EnvironmentLoader).processField(0xc8201fe880, 0xadd680, 0xc, 0xc8201dc6c0, 0xad1120, 0xa, 0x8fdba0, 0xc8201d2090, 0x0, 0x0)
        /home/jonaz/go/src/github.com/koding/multiconfig/env.go:61 +0x1da
    github.com/koding/multiconfig.(*EnvironmentLoader).Load(0xc8201fe880, 0x8d2160, 0xc8201cb3b0, 0x0, 0x0)
        /home/jonaz/go/src/github.com/koding/multiconfig/env.go:45 +0x1bd
    github.com/koding/multiconfig.multiLoader.Load(0xc8201f2fc0, 0x4, 0x4, 0x8d2160, 0xc8201cb3b0, 0x0, 0x0)
        /home/jonaz/go/src/github.com/koding/multiconfig/multiloader.go:14 +0xc6
    github.com/koding/multiconfig.(*multiLoader).Load(0xc8201fe8a0, 0x8d2160, 0xc8201cb3b0, 0x0, 0x0)
        <autogenerated>:7 +0xc1
    github.com/stampzilla/stampzilla-go/nodes/stampzilla-server/notifications.(*router).Load(0xc8201d20f0, 0xb3b9a0, 0x12, 0x0, 0x0)
        /home/jonaz/go/src/github.com/stampzilla/stampzilla-go/nodes/stampzilla-server/notifications/router.go:49 +0x210
    
    

    multiconfig is used here: https://github.com/stampzilla/stampzilla-go/blob/master/nodes/stampzilla-server/notifications/router.go#L49

    If you need more than that please say and i'll put together a runnable example.

  • Allow configuration to be specified as an io.Reader

    Allow configuration to be specified as an io.Reader

    Rather than just a path on disk

    This allows for things like reading the configuration from STDIN

    First of all, awesome package! I like that you put composability first.

    I'm not super crazy about the implementation of this PR, but the existing structs are very centered around reading from files so I'm not sure what the best way to handle adding something like this is. If you are open to changing the interface of the package, I'd recommend centering the API around io.Readers and adding helper functions for dealing with files.

    Anyway, I just wanted to get your thoughts about adding something like this to the package.

  • env usage help prints env vars in random order

    env usage help prints env vars in random order

    from env.go

    iteration over a map is random. Need to copy into the []string, sort, then then use that for range.

    // PrintEnvs prints the generated environment variables to the std out.
    func (e *EnvironmentLoader) PrintEnvs(s interface{}) {
        strct := structs.New(s)
        strctMap := strct.Map()
        prefix := e.getPrefix(strct)
    
        for key, val := range strctMap {
            field := strct.Field(key)
    
            e.printField(prefix, field, key, val)
        }
    }
    
  • Toml cannot be used map

    Toml cannot be used map

    #config.toml 
    title       = "blabla"
    [servers]
        [servers.a]
        ip      = "1.1.1.1"
        dc      = "abc"
        [servers.b]
        ip      = "2.2.2.2"
        dc      = "cba"
    
    
    #main.go
    type (
    	WebSiteConfig struct {
    		Title string
    		Servers map[string]Server
    	}
    )
    
    type Server struct {
    		IP string
    		DC string
    }
    
    func Load() *WebSiteConfig {
    	config := new(WebSiteConfig)
    	multiconfig.NewWithPath(LogPath). MustLoad(config)
    
    	return config
    }
    
    
    #error 
    panic: not struct
    
    goroutine 1 [running]:
    github.com/fatih/structs.strctVal(0xc0001c4500, 0xc0001b88f8, 0x16, 0xc0001c4500, 0xc0001b88f8)
            /Users/Xwei/go/src/github.com/fatih/structs/structs.go:437 +0x12f
    github.com/fatih/structs.(*Field).FieldOk(0xc0002122d0, 0xc0002143c2, 0x1, 0xc0001ddc80, 0xc0001caa10)
            /Users/Xwei/go/src/github.com/fatih/structs/field.go:129 +0x98
    github.com/fatih/structs.(*Field).Field(0xc0002122d0, 0xc0002143c2, 0x1, 0x157ad93)
            /Users/Xwei/go/src/github.com/fatih/structs/field.go:109 +0x3f
    github.com/koding/multiconfig.(*EnvironmentLoader).processField(0xc0001c02a0, 0x1596ef3, 0xd, 0xc0002122d0, 0x157ad93, 0x7, 0x15fa320, 0xc0001b93b0, 0x0, 0x0)
            /Users/Xwei/go/src/github.com/koding/multiconfig/env.go:61 +0x164
    github.com/koding/multiconfig.(*EnvironmentLoader).Load(0xc0001c02a0, 0x15c80e0, 0xc0001b88d0, 0x0, 0x0)
    
    
    
  • bug : flag: help requested

    bug : flag: help requested

    code: m := multiconfig.NewWithPath(configPath) config := new(common.Configs) m.MustLoad(config)

    app -help Usage of app: ... ... flag: help requested

    This panic arises func (d *DefaultLoader) MustLoad(conf interface{})

    Want to know why

  • Not able to specify configuration file on command line

    Not able to specify configuration file on command line

    Unless I'm missing something obvious, I can't find a way to specify configuration file on command line like app -conf config.toml

    Using the flags package on top of multiconfig breaks the helpfile and produces a "flag provided but not defined".

  • DefaultLoader/FlagLoader is unusable

    DefaultLoader/FlagLoader is unusable

    A common case for production is you need to use a command line parameter to specify the path of config file, then read and parse it. But that parameter option is apparently not included in FlagLoader's FlagSet, which will result in

    flag provided but not defined
    

    error.

    Need to remove it from DefaultLoader.

Cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration

cfginterpolator cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration cfginterpolat

Dec 14, 2021
Jul 4, 2022
⚙️ Dead Simple Config Management, load and persist config without having to think about where and how.

Configo Dead Simple Config Management, load and persist config without having to think about where and how. Install go get github.com/UltiRequiem/conf

Apr 6, 2022
Quick and easy way to load config files based on a simple set of rules.
Quick and easy way to load config files based on a simple set of rules.

config Quick and easy way to load config files based on a simple set of rules. Project inspired by https://github.com/lorenwest/node-config Important

Apr 9, 2021
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Jan 8, 2023
12 factor configuration as a typesafe struct in as little as two function calls

Config Manage your application config as a typesafe struct in as little as two function calls. type MyConfig struct { DatabaseUrl string `config:"DAT

Dec 13, 2022
JSON or YAML configuration wrapper with convenient access methods.

Config Package config provides convenient access methods to configuration stored as JSON or YAML. This is a fork of the original version. This version

Dec 16, 2022
Configure is a Go package that gives you easy configuration of your project through redundancy

Configure Configure is a Go package that gives you easy configuration of your project through redundancy. It has an API inspired by negroni and the fl

Sep 26, 2022
An opinionated configuration loading framework for Containerized and Cloud-Native applications.
An opinionated configuration loading framework for Containerized and Cloud-Native applications.

Opinionated configuration loading framework for Containerized and 12-Factor compliant applications. Read configurations from Environment Variables, an

Dec 16, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Nov 3, 2022
A minimalist Go configuration library
A minimalist Go configuration library

fig fig is a tiny library for loading an application's config file and its environment into a Go struct. Individual fields can have default values def

Dec 23, 2022
go-up! A simple configuration library with recursive placeholders resolution and no magic.

go-up! A simple configuration library with placeholders resolution and no magic. go-up provides a simple way to configure an application from multiple

Nov 23, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
Go configuration made easy!

gofigure Go configuration made easy! Just define a struct and call Gofigure Supports strings, ints/uints/floats, slices and nested structs Supports en

Sep 26, 2022
Harvest configuration, watch and notify subscriber

Harvester Harvester is a configuration library which helps setting up and monitoring configuration values in order to dynamically reconfigure your app

Dec 26, 2022
go implementation of lightbend's HOCON configuration library https://github.com/lightbend/config

HOCON (Human-Optimized Config Object Notation) Configuration library for working with the Lightbend's HOCON format. HOCON is a human-friendly JSON sup

Dec 3, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022