Small library to read your configuration from environment variables

envconfig

CI Go Reference

envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct.

See the example to understand how to use it, it's pretty simple.

Supported types

  • Almost all standard types plus time.Duration are supported by default.
  • Slices and arrays
  • Arbitrary structs
  • Custom types via the Unmarshaler interface.

How does it work

envconfig takes the hierarchy of your configuration struct and the names of the fields to create a environment variable key.

For example:

var conf struct {
    Name string
    Shard struct {
        Host string
        Port int
    }
}

This will check for those 3 keys:

  • NAME or name
  • SHARD_HOST, or shard_host
  • SHARD_PORT, or shard_port

Flexible key naming

envconfig supports having underscores in the key names where there is a word boundary. Now, that term is not super explicit, so let me show you an example:

var conf struct {
    Cassandra struct {
        SSLCert string
        SslKey string
    }
}

This will check all of the following keys:

  • CASSANDRA_SSL_CERT, CASSANDRA_SSLCERT, cassandra_ssl_cert, cassandra_sslcert
  • CASSANDRA_SSL_KEY, CASSANDRA_SSLKEY, cassandra_ssl_key, cassandra_sslkey

If that is not good enough, look just below.

Custom environment variable names

envconfig supports custom environment variable names:

var conf struct {
    Name string `envconfig:"myName"`
}

Default values

envconfig supports default values:

var conf struct {
    Name string `envconfig:"default=Vincent"`
}

Optional values

envconfig supports optional values:

var conf struct {
    Name string `envconfig:"optional"`
}

Skipping fields

envconfig supports skipping struct fields:

var conf struct {
    Internal string `envconfig:"-"`
}

Combining multiple options in one tag

You can of course combine multiple options:

var conf struct {
    Name string `envconfig:"default=Vincent,myName"`
}

Slices or arrays

With slices or arrays, the same naming is applied for the slice. To put multiple elements into the slice or array, you need to separate them with a , (will probably be configurable in the future, or at least have a way to escape)

For example:

var conf struct {
    Ports []int
}

This will check for the key PORTS:

  • if your variable is 9000 the slice will contain only 9000
  • if your variable is 9000,100 the slice will contain 9000 and 100

For slices of structs, it's a little more complicated. The same splitting of slice elements is done with a comma, however, each token must follow a specific format like this: {<first field>,<second field>,...}

For example:

var conf struct {
    Shards []struct {
        Name string
        Port int
    }
}

This will check for the key SHARDS. Example variable content: {foobar,9000},{barbaz,20000}

This will result in two struct defined in the Shards slice.

If you want to set default value for slice or array, you have to use ; as separator, instead of ,:

var conf struct {
    Ports []int `envconfig:"default=9000;100"`
}

Same for slices of structs:

var conf struct {
    Shards []struct {
        Name string
        Port int
    } `envconfig:"default={foobar;localhost:2929};{barbaz;localhost:2828}"`
}

Development state

I consider envconfig to be pretty much done.

It has been used extensively at Batch for more than 5 years now without much problems, with no need for new features either.

So, while I will keep maintaining this library (fixing bugs, making it compatible with new versions of Go and so on) for the foreseeable future, I don't plan on adding new features.

But I'm open to discussion so if you have a need for a particular feature we can discuss it.

Owner
Vincent Rischmann
Still learning
Vincent Rischmann
Comments
  • Handling embedded structs - should probably allow shorter name?

    Handling embedded structs - should probably allow shorter name?

    Hi! Thanks for this useful tool. I have minor suggestion (happy to do PR as well). Currently this fails

    package main
    
    import (
        "fmt"
        "os"
    
        "github.com/vrischmann/envconfig"
    )
    
    type OtherConfig struct {
        Address string
    }
    
    type Config struct {
        OtherConfig
    }
    
    func main() {
        os.Setenv("CONFIG_ADDRESS", "example.com")
        var conf struct {
            Config
        }
        if err := envconfig.Init(&conf); err != nil {
            fmt.Printf("err=%s\n", err)
        }
    }
    

    with

    err=envconfig: keys CONFIG_OTHERCONFIG_ADDRESS, CONFIG_OTHER_CONFIG_ADDRESS, config_other_config_address, config_otherconfig_address not found
    

    I think it should be reasonable to include CONFIG_ADDRESS in env var names searched. This is also inline with Go semantics where you can write embedded fields without specifying full path, e.g.

        conf.Config.Address = "123"
        conf.Config.OtherConfig.Address = "123"
    

    both work.

    WDYT?

  • Slices are cleared and appended, should be overwritten

    Slices are cleared and appended, should be overwritten

    Hi

    I've a scenario, where at first configuration struct is loaded from config file and then envconfig is applied to overwrite some settings from environment variables, when they exist.

    One of the fields in the config struct is a slice of string. When an environment variable for this slice exists, the slice is not replaced by value from the environment variable. Instead, resulting slice contains as many items as it was in config file set to empty string followed by environment variable value.

    For example:

    • original slice value from config file - ["foo"]
    • environment variable value - "bar"
    • slice after envconfig - ["", "bar"]

    I guess the bug is in function setSliceField on line https://github.com/vrischmann/envconfig/blob/e689cc56e3bd23fb94f1caea0a526c60cfff085c/envconfig.go#L252

    There shouldn't be value.Len() but 0 in my opinion.

    Best Regards

    Pavel

  • Tag new release from master

    Tag new release from master

    If this project truly is "done", could you please tag a new release now that #20 has been merged? This is making for some awkward dependency management on our end. Thanks.

  • ensure variables are unset before testing

    ensure variables are unset before testing

    Without this, tests fail if one variable is already set, e.g.:

    $ export NAME=2
    $ go test ./...
    envconfig_test.go:25:
    
        exp: "envconfig: key NAME not found"
    
        got: "envconfig: key LOG_PATH not found"
    
    --- FAIL: TestParseSimpleConfig (0.00s)
    FAIL
    FAIL    github.com/vrischmann/envconfig 0.006s
    
  • [Feature request] Print all unset environment variable

    [Feature request] Print all unset environment variable

    Description

    Right now parsing the environment variable process is failing fast. That's mean that only first uninitialized env is printed. If you have 5 environments variable then you can even need to execute the application 5 times to get info about all variables. I know that every application should have proper documentation but still, IMO it will be super useful to have an option to change that behavior and print all unset environment right after the first execution.

    Details

    Assumption that app has the following config:

    type Config struct {
    	Debug             bool
    	ClientID          string
    }
    

    and user didn't provide both envs.

    Current solution Output:

    envconfig: keys DEBUGBB, DEBUG_BB, debug_bb, debugbb not found
    

    Desired solution

    Add new option param

    	envconfig.InitWithOptions(&conf, envconfig.Options{
    		PrintAllUnsetEnvs: true,
    	})
    

    Output:

    envconfig:
    	* key DEBUG not found
    	* keys CLIENTID, CLIENT_ID, client_id, clientid not found
    

    I you will agree with me then I can also provide the implementation for that.

  • add optional global config value for each parse

    add optional global config value for each parse

    Great package! this is nearly perfect for my needs. I'm hoping to negotiate a way to make not finding keys not throw errors by default. of course, can fork, but would prefer to contribute to the main line (may not be the only one). The rationale / motivation being that I basically want to just add this to parse pre-existing configs without having to add optionals in each of the config lines (which may later change to end up not being used, etc), and for many of those fields I do not care whether they exist or not. Basically, allowing the application to do any validations (we have many pre-existing with "default" rewriting already externally from this package). I think providing both in the package is super useful, I would just like to be able to opt out. :)

    rationale for doing the change in this specific way is to not break the API (which is great -- stupid simple) or add more Init-esque functions. Of course, global config var is not ideal, but am very open to discussion of alternatives and am happy to contribute any conclusions we come to.

    client usage (if not obvious):

    var config Config
    envconfig.Optional = true
    envconfig.Init(&config)
    

    thank you!

  • Marshal to map[string]string

    Marshal to map[string]string

    it would be interesting to make this bidirectional. For example, could I generate environment variables from a structure (and return as map[string]string):

    v := struct {
      User string
      Pass string
    }{ "root", "pa55word" }
    
    envs, err := envconfig.Marshal(&v)
    envs, err := envconfig.MarshalPrefix("MYSQL", &v)
    
    // where envs looks like this
    map[string]string {
      "MYSQL_USER": "root",
      "MYSQL_PASS": "pa55word",
    }
    
  • optional bool should not throw exception if empty

    optional bool should not throw exception if empty

    I have a boolean field marked as optional:

    type Repo struct {
        Private bool `envconfig:"optional"
    }
    

    Even thought it is optional I still receive a parse error:

    strconv.ParseBool: parsing "": invalid syntax
    

    This pull request attempts to ignore and use default values for optional fields with empty strings.

  • [Feature Request] default tag support for slices

    [Feature Request] default tag support for slices

    Currently default tag is unsupported on slices. As it explained here:

    The default tag is unsupported on slices because slice parsing uses , as the separator, as does the envconfig tags separator.

    What I propose is to change default tag to support following format:

    default=value1;value2;etc
    

    If this value will be applied to slice, it will be split by ;. If applied to string, value is applied as is. Else, it will panic. I would gladly implement this feature.

  • add options struct to initialization

    add options struct to initialization

    also slightly changes behavior such that unexported fields no longer cause the program to panic (we skip looking for them in the environment, as well).

    Options struct allows three new options to be set on Init.

    1. AllOptional will force the parse to consider each field optional by default, rather than erroring when it comes across a non-optional field that is empty.

    2. LeaveNil will make the parse leave pointers in a nil state if no inner fields or values are found after coming across a nil pointer. If it does come across any non-empty values, then the behavior is to create a new pointer and fill in that field. This works for any arbitrary depth, all parent pointers will be created as needed, no matter how deeply nested the non-empty value is.

    3. AllowUnexported allows for unexported fields to be present inside of the config. We will not set or look for any unexported fields, they will simply be skipped over.

    some discussion from #12 (continued here)

    agree, went for the options struct and think that is a good choice. already found 3 uses for it :)

    I think this is much better. I added some more options that are useful for my use case, turned off by default. Happy to add any more necessary clarification or answer questions.

    Thank you!

  • encoding byte slice similar to json encoder

    encoding byte slice similar to json encoder

    Many Go encoding libraries, including the encoding/json, encode []byte to base64: http://golang.org/src/encoding/json/decode.go?s=19000:19540#L741

    This patch modifies this package to behave in the same manner as the encoding/json package and expects byte arrays to be base64 encoded. I think having consistency among decoders would be nice, and it seems (to me) a base64 string is a bit easier to construct than a comma-separated slice of unit8, especially since posix environments ship with the base64 --encode command line utility.

    This is just a proposal, of course, and I completely understand if you choose not to merge.

  • Add option to prepend prefix also to custom name taken from tag

    Add option to prepend prefix also to custom name taken from tag

    By default the struct like:

    type Config struct {
    	TestEnvFoo string `envconfig:"TEST_ENV_FOO"`
    	TestEnvBar string
    }
    

    can be initialised from env: TEST_ENV_FOO, TEST_ENV_BAR if no prefix specified, but from TEST_ENV_FOO, PFX_TEST_ENV_BAR, if PFX prefix is set.

    I added ability to enable consistent prefix addition (see tests), with:

    envconfig.InitWithOptions(&conf, envconfig.Options{
    	Prefix:    "PFX",
    	PrefixTag: true,
    })
    

    the struct described above will match PFX_TEST_ENV_FOO, PFX_TEST_ENV_BAR envs.

    The default behaviour remain unchanged.

Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

May 11, 2021
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Nov 12, 2021
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

Aug 22, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Dec 30, 2021
formicidate is a small tool for Go application can update the value of environment variables in a .env file with code

formicidae Update .env files in Go with code. What is fomicidae? formicidate is a small tool for Go application. You can update the value of environme

Jan 23, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

Jan 5, 2023
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Jan 2, 2023
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Jan 9, 2023
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Sep 26, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Sep 26, 2022
Environment variables substitution for Go

envsubst Environment variables substitution for Go. see docs below Installation: From binaries Latest stable envsubst prebuilt binaries for 64-bit Lin

Jan 1, 2023
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

Dec 3, 2021
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

Apr 11, 2022
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Jan 8, 2023
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Dec 29, 2022
Cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration

cfginterpolator cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration cfginterpolat

Dec 14, 2021