✨Clean and minimalistic environment configuration reader for Golang

Clean Env

Clean Env

Minimalistic configuration reader

Mentioned in Awesome Go GoDoc Go Report Card Coverage Status Build Status Release License

Overview

This is a simple configuration reading tool. It just does the following:

  • reads and parses configuration structure from the file
  • reads and overwrites configuration structure from environment variables
  • writes a detailed variable list to help output

Content

Installation

To install the package run

go get -u github.com/ilyakaznacheev/cleanenv

Usage

The package is oriented to be simple in use and explicitness.

The main idea is to use a structured configuration variable instead of any sort of dynamic set of configuration fields like some libraries does, to avoid unnecessary type conversions and move the configuration through the program as a simple structure, not as an object with complex behavior.

There are just several actions you can do with this tool and probably only things you want to do with your config if your application is not too complicated.

  • read configuration file
  • read environment variables
  • read some environment variables again

Read Configuration

You can read a configuration file and environment variables in a single function call.

import github.com/ilyakaznacheev/cleanenv

type ConfigDatabase struct {
    Port     string `yaml:"port" env:"PORT" env-default:"5432"`
    Host     string `yaml:"host" env:"HOST" env-default:"localhost"`
    Name     string `yaml:"name" env:"NAME" env-default:"postgres"`
    User     string `yaml:"user" env:"USER" env-default:"user"`
    Password string `yaml:"password" env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This will do the following:

  1. parse configuration file according to YAML format (yaml tag in this case);
  2. reads environment variables and overwrites values from the file with the values which was found in the environment (env tag);
  3. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

Read Environment Variables Only

Sometimes you don't want to use configuration files at all, or you may want to use .env file format instead. Thus, you can limit yourself with only reading environment variables:

import github.com/ilyakaznacheev/cleanenv

type ConfigDatabase struct {
    Port     string `env:"PORT" env-default:"5432"`
    Host     string `env:"HOST" env-default:"localhost"`
    Name     string `env:"NAME" env-default:"postgres"`
    User     string `env:"USER" env-default:"user"`
    Password string `env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadEnv(&cfg)
if err != nil {
    ...
}

Update Environment Variables

Some environment variables may change during the application run. To get the new values you need to mark these variables as updatable with the tag env-upd and then run the update function:

import github.com/ilyakaznacheev/cleanenv

type ConfigRemote struct {
    Port     string `env:"PORT" env-upd`
    Host     string `env:"HOST" env-upd`
    UserName string `env:"USERNAME"`
}

var cfg ConfigRemote

cleanenv.ReadEnv(&cfg)

// ... some actions in-between

err := cleanenv.UpdateEnv(&cfg)
if err != nil {
    ...
}

Here remote host and port may change in a distributed system architecture. Fields cfg.Port and cfg.Host can be updated in the runtime from corresponding environment variables. You can update them before the remote service call. Field cfg.UserName will not be changed after the initial read, though.

Description

You can get descriptions of all environment variables to use them in the help documentation.

import github.com/ilyakaznacheev/cleanenv

type ConfigServer struct {
    Port     string `env:"PORT" env-description:"server port"`
    Host     string `env:"HOST" env-description:"server host"`
}

var cfg ConfigRemote

help, err := cleanenv.GetDescription(&cfg, nil)
if err != nil {
    ...
}

You will get the following:

Environment variables:
  PORT  server port
  HOST  server host

Model Format

Library uses tags to configure the model of configuration structure. There are the following tags:

  • env="<name>" - environment variable name (e.g. env="PORT");
  • env-upd - flag to mark a field as updatable. Run UpdateEnv(&cfg) to refresh updatable variables from environment;
  • env-required - flag to mark a field as required. If set will return an error during environment parsing when the flagged as required field is empty (default Go value). Tag env-default is ignored in this case;
  • env-default="<value>" - default value. If the field wasn't filled from the environment variable default value will be used instead;
  • env-separator="<value>" - custom list and map separator. If not set, the default separator , will be used;
  • env-description="<value>" - environment variable description;
  • env-layout="<value>" - parsing layout (for types like time.Time);

Supported types

There are following supported types:

  • int (any kind);
  • float (any kind);
  • string;
  • boolean;
  • slices (of any other supported type);
  • maps (of any other supported type);
  • time.Duration;
  • time.Time (layout by default is RFC3339, may be overridden by env-layout);
  • any type implementing cleanenv.Setter interface.

Custom Functions

To enhance package abilities you can use some custom functions.

Custom Value Setter

To make custom type allows to set the value from the environment variable, you need to implement the Setter interface on the field level:

type MyField string

func (f *MyField) SetValue(s string) error  {
    if s == "" {
        return fmt.Errorf("field value can't be empty")
    }
    *f = MyField("my field is: "+ s)
    return nil
}

type Config struct {
    Field MyField `env="MY_VALUE"`
}

SetValue method should implement conversion logic from string to custom type.

Custom Value Update

You may need to execute some custom field update logic, e.g. for remote config load.

Thus, you need to implement the Updater interface on the structure level:

type Config struct {
    Field string
}

func (c *Config) Update() error {
    newField, err := SomeCustomUpdate()
    f.Field = newField
    return err
}

Supported File Formats

There are several most popular config file formats supported:

  • YAML
  • JSON
  • TOML
  • ENV
  • EDN

Integration

The package can be used with many other solutions. To make it more useful, we made some helpers.

Flag

You can use the cleanenv help together with Golang flag package.

// create some config structure
var cfg config 

// create flag set using `flag` package
fset := flag.NewFlagSet("Example", flag.ContinueOnError)

// get config usage with wrapped flag usage
fset.Usage = cleanenv.FUsage(fset.Output(), &cfg, nil, fset.Usage)

fset.Parse(os.Args[1:])

Examples

type Config struct {
    Port string `yaml:"port" env:"PORT" env-default:"8080"`
    Host string `yaml:"host" env:"HOST" env-default:"localhost"`
}

var cfg Config

err := ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This code will try to read and parse the configuration file config.yml as the structure is described in the Config structure. Then it will overwrite fields from available environment variables (PORT, HOST).

For more details check the example directory.

Contribution

The tool is open-sourced under the MIT license.

If you will find some error, want to add something or ask a question - feel free to create an issue and/or make a pull request.

Any contribution is welcome.

Thanks

Big thanks to a project kelseyhightower/envconfig for inspiration.

The logo was made by alexchoffy.

Blog Posts

Clean Configuration Management in Golang.

Owner
Ilya Kaznacheev
Backend SWE. Founder of Golang Voronezh, SAP Community Voronezh, active speaker & community member.
Ilya Kaznacheev
Comments
  • Problem with bool variable in YAML file

    Problem with bool variable in YAML file

    yaml file

    isdebug: false
    rule:
      cachesize: 10485760000
    

    golang struct

    type AppConfig struct {
    	IsDebug bool `env:"RE_IS_DEBUG" env-required:"true"`
    	Rule struct {
    		CacheSize          int    `env:RULE_CACHE_SIZE" env-required:"true"`
            }
    

    i got an error field \"IsDebug\" is required but the value is not provided", but if set isdebug: true - all ok, config parse without any errors.

  • BUG: env-default overwrites value from yml file

    BUG: env-default overwrites value from yml file

    Wehnever env-default tag is defined, it's value overwrites value read from yml file (which is not intended behaviour, according to https://github.com/ilyakaznacheev/cleanenv#read-configuration )

    100% reproducibility, observed while running go app in vagrant box (ubuntu/bionic).

    configs/config.yml

    # server config
    server:
      host: "localhost"
      port: 3000
    
    # database config
    db:
      type: "mongodb"
      mongoConfig:
                # full endpoint: address + port
        URL: "mongodb://127.0.0.1:27017"
        ProjectsDBName: "Projects"
        SchemasDBName: "Schemas"
    

    config.go

    package common
    package common
    
    import (
      "github.com/ilyakaznacheev/cleanenv"
      "fmt"
    )
    
    const YmlConfFile string = "./configs/config.yml"
    
    type AppConfig struct {
    	ServerConfig struct {
    		PORT string `yaml:"port" env:"MYAPP_PORT" env-default:"3000" env-upd`
    		HOST string `yaml:"host" env:"MYAPP_HOST" env-default:"localhost" env-upd`
    	} `yaml:"server"`
    	DBConfig struct {
    		DBType string `yaml:"type" env:"MYAPP_DB_SRV" env-default:"mongodb"`
    		MongoConfig struct {
    			URL string			`yaml:"URL" env:"MYAPP_DB_URL" env-default:"mongodb://localhost:27017" env-upd`
    			ProjectsDB string	`yaml:"ProjectsDBName" env:"MYAPP_PROJECTS_DB" env-default:"TProjects" env-upd`
    			SchemasDB string	`yaml:"SchemasDBName"  env:"MYAPP_SCHEMAS_DB" env-default:"TSchemas" env-upd`
    		} `yaml:"mongoConfig"`
    	} `yaml:"db"`
    }
    
    func NewAppConfig(confFilePath string) *AppConfig {
    	var AppC AppConfig
        err := cleanenv.ReadConfig(confFilePath, &AppC)
        if err != nil { panic(err) }
    	fmt.Println("AppC.DBConfig.MongoConfig.URL %v",AppC.DBConfig.MongoConfig.URL)
    	return &AppC
    }
    

    main.go

    package main
    import "myappp/common"
    
    func main() {
      appConfig := common.NewAppConfig(common.YmlConfFile)
    }
    

    it prints: AppC.DBConfig.MongoConfig.URL %v mongodb://localhost:27017 (as in env-default)

    when I remove respective env-default tag with value, e.g.

    it prints

    AppC.DBConfig.MongoConfig.URL %v mongodb://127.0.0.1:27017 (as taken from file)

    Aside from report above, let me mention that's a great effort to conveniently handle different configuration sources. Thank you!

  • Expand environment variables when reading from file

    Expand environment variables when reading from file

    Hi,

    it would be cool if the library would support expansion of environment variables using os.ExpandEnv A use-case for this feature would be if you need to construct a configuration variable based on multiple different ones. e.g. building an URL wrong multiple environment variables

    I've tried to figure out where this must be added but couldn't find the write code part.

    Thanks

    PS: i'm thinking about something like this https://mtyurt.net/post/go-using-environment-variables-in-configuration-files.html

  • Updating example app to include .env config loading

    Updating example app to include .env config loading

    The example was lacking demonstration of how environment variables can override the yaml configs. I have added a .env file and using godotenv to load them first.

  • Can not overwrite variable

    Can not overwrite variable

    I'm having hard time overrriding a value from enviroment variable. Here is what I want. I want to read config from a enviroment file. Then read enviroment varialbes and overwrite them even If they exist in the config file. The following code does not overwrite value from enviroment varialbe as it supposed to:

    
    type ConfigDatabase struct {
    		Port     string `env:"PORT" env-default:"5432"`
    		Host     string `env:"HOST" env-default:"localhost"`
    		Name     string `env:"NAME" env-default:"postgres"`
    		User     string `env:"USER" env-default:"user"`
    		Password string `env:"PASSWORD"`
    	}
    
    	var cfg ConfigDatabase
    	err := cleanenv.ReadConfig("config.env", &cfg)
    	if err != nil {
    		log.Println(err)
    	}
    
    	err1 := cleanenv.ReadEnv(&cfg)
    	if err1 != nil {
    		log.Println(err1)
    	}
    	fmt.Println(cfg)
    
    

    Even witih cleanenv.UpdateEnv It still does not work. To my understanding, Each Read* function should overwrite the previous values.

    Also the env-update struct tag is not working either.

    
    Password string `env:"PASSWORD" env-upd`
    
    

    I get an warning in VSCode struct field tagenv:"PASSWORD" env-updnot compatible with reflect.StructTag.Get: bad syntax for struct tag pair

  • Add url support

    Add url support

    First off, thanks for this package!

    I wanted to use it in our project, but we have in our environment a few URLs (DB connection strings in a single value, or URL for an image CDN for example), which should be valid ones. It felt to me a fairly valid case to have as supported type, so here's a PR for it!

  • Value containing `.` always clipped

    Value containing `.` always clipped

    Trying to read a value that contains . and it doesn't matter what I set the env-separator to. It gets clipped anyway.

    Am I using it wrong or this a bug?

  • Please fix readme

    Please fix readme

    Readme examples don't work out-of-the-box.

    Readme suggests using 'yml' tag that doesn't seem to work. Tests use 'yaml' instead.

    Setter from readme might be invalid as well. It should be defined on pointer type.

    Reproduced on go1.13 darwin/amd64

  • Any plans to move to yaml v3?

    Any plans to move to yaml v3?

    I would like to use yaml.Node in my config files but have found they don't work with cleanenv. As an experiment, I changed the import in cleanenv.go to use yaml v3 instead and things with yaml.Node started working. I did no testing otherwise though.

    So, I'm curious, any plans to move to yaml v3?

  • Update library used for toml support

    Update library used for toml support

    The currently used toml library (github.com/BurntSushi/toml) has officially been marked as unmaintained by its owner. As such, it might be wise to move to a new library (support for newer versions of the TOML spec as burntsushi/toml only supports v0.4.0, bug/security fixes, etc).

  • Use config value over default if not in env

    Use config value over default if not in env

    This is a proposed fix for https://github.com/ilyakaznacheev/cleanenv/issues/40.

    This behaves the way I expected, given this line of the documentation:

    1. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

    in https://github.com/ilyakaznacheev/cleanenv#read-configuration.

    I'm no reflect wizard, so I suggest a careful review. It works in my testing though.

  • Add support of nested struct in map store by pointer

    Add support of nested struct in map store by pointer

    Added supporting of nested struct in map stored by pointer.

    I don't see a way to support structs in maps stored by value, because we can't directly change structs in maps. Perhaps this requires a major change in library logic.

    Refers #49

  • Does this support parsing `map[string]CustomStruct` using the `env` tag?

    Does this support parsing `map[string]CustomStruct` using the `env` tag?

    For example, I have the following Config root

    
    var RootConfig Root
    
    type Root struct {
    	Http map[string]HttpConfig `yaml:"http" env:"HTTP"`
    }
    
    type HttpConfig struct {
    	Host      string `yaml:"host"`
    	TimeoutMs int    `yaml:"timeout_ms"`
    }
    
    func Load(configPath string) {
    	cleanenv.ReadConfig(configPath, &RootConfig)
    }
    

    I know I can provide the following to populate http key via the following yaml structure:

    http:
      payment:
        host: "http://payment:4000"
        timeout_ms: 2000
    

    But what about the via environment variable? What would HTTP env var look like?

  • Apply Custom Value Setters to custom struct types

    Apply Custom Value Setters to custom struct types

    I realized that a SetValue() defined on a custom struct type is never called. Insted, the struct is flattened. This prevents me from parsing e.g., a yaml string from an ENV variable:

    type Config struct {
    	Complex ComplexYamlConfig `env:"COMPLEX_CONFIG"`
    }
    
    type ComplexYamlConfig struct {
    	Left  string `yaml:"myProp"`
    }
    
    func (c *ComplexYamlConfig) SetValue(s string) error {
            // will never be called by cleanenv
    	return yaml.Unmarshal([]byte(s), c)
    }
    

    Is this something that can be added? Or is there a different way of achieving the same thing?

  • Didn't skip unexported field

    Didn't skip unexported field

    @ilyakaznacheev

    panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
    
    goroutine 1 [running]:
    reflect.valueInterface({0x95f8a0?, 0xc00055c248?, 0xabea8f?}, 0xa?)
            E:/software/dev_core/go1.18.1/src/reflect/value.go:1441 +0xd8
    reflect.Value.Interface(...)
            E:/software/dev_core/go1.18.1/src/reflect/value.go:1430
    github.com/ilyakaznacheev/cleanenv.readStructMetadata({0x9c7900?, 0xc00055c1c0?})
    
            E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:274 +0x771
    github.com/ilyakaznacheev/cleanenv.readEnvVars({0x9c7900, 0xc00055c1c0}, 0x0)
    
            E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:333 +0x45
    github.com/ilyakaznacheev/cleanenv.ReadConfig({0xace493?, 0xc00041f9a8?}, {0x9c7900, 0xc00055c1c0})
    
    
            E:/text/code-project/GolangProject/pkg/mod/github.com/ilyakaznacheev/[email protected]/cleanenv.go:95 +0x48
    

    It doesn't skip unexported field

    Originally posted by @838239178 in https://github.com/ilyakaznacheev/cleanenv/issues/68#issuecomment-1279943050

    Example

    package main
    
    import (
    	"github.com/ilyakaznacheev/cleanenv"
    )
    
    type Children struct {
    	name string `yaml:"name" env:"name"`
    }
    
    type Config struct {
    	private Children `yaml:"private" env-prefix:"private"`
    	Public string `yaml:"public" env:"public"`
    }
    
    func main()  {
    	var conf Config
    	if err := cleanenv.ReadConfig("config.yml", &conf); err != nil {
    		panic(err)
    	}
    }
    
  • Read From Environment Variable Even If Parse File Failed

    Read From Environment Variable Even If Parse File Failed

    Related Issue

    https://github.com/ilyakaznacheev/cleanenv/issues/79

    Motivation and Context

    Full Snippet of Read Configuration section of README

    You can read a configuration file and environment variables in a single function call.

    import "github.com/ilyakaznacheev/cleanenv"
    
    type ConfigDatabase struct {
        Port     string `yaml:"port" env:"PORT" env-default:"5432"`
        Host     string `yaml:"host" env:"HOST" env-default:"localhost"`
        Name     string `yaml:"name" env:"NAME" env-default:"postgres"`
        User     string `yaml:"user" env:"USER" env-default:"user"`
        Password string `yaml:"password" env:"PASSWORD"`
    }
    
    var cfg ConfigDatabase
    
    err := cleanenv.ReadConfig("config.yml", &cfg)
    if err != nil {
        ...
    }
    

    This will do the following:

    1. parse configuration file according to YAML format (yaml tag in this case);
    2. reads environment variables and overwrites values from the file with the values which was found in the environment (env tag);
    3. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

    Current behavior of ReadConfig is that it will return an error if it failed to read configuration from file and the method will not attempt to read configuration from environment variables. Thus, this statement from Read Configuration section of README is not necessarily true:

    reads environment variables and overwrites values from the file with the values which was found in the environment (env tag);

    Because parseFile does not fill the config struct with default values and readEnvVars is the method that will fill the config struct with default values, once parseFile returns an error ReadConfig will not execute readEnvVars. Which causes the current behavior of ReadConfig invalidating this statement:

    if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

    To address the discrepancy, we decided to open this PR to update ReadConfig so that the method:

    • Will continue to attempt to read configuration from environment variables even if it failed to read configuration from file
    • Only returns an error if it can't read configuration from both file and environment variable
    • Fill config struct with default values even if parseFile or readEnvVars returns an error

    How Has This Been Tested?

    All preexisting tests passed except unknown subtest and parsing error subtest for TestReadConfig due to the behavior change we implemented for ReadConfig. Thus we modified unknown and parsing error to become unknown filetype fallback to env and parsing error fallback to env respectively. So that instead of testing to ensure ReadConfig returns an error if it failed to parse file, the updated test cases testing to ensure ReadConfig will run readEnvVars if it failed to parse file.

    Two new subtests were added to TestReadConfig. The first one, called file and env parse error, was added to cover the case where neither the file or environment variables could be successfully read. The second one, called parsed file and failed to parse env, was added to cover the case where ReadConfig is able to read from file but not able to read from environment variables.

    Since readDefaults method was added to prepopulate the config struct with default values, new test called TestReadDefaults, with two subtests, was added to ensure the method is properly tested.

    Besides ReadConfig and the newly created readDefaults, functionalities of other methods were not modified. Thus, changes implemented in this PR should not affect other areas of this library.

    Checklist:

    • [x] My code follows the code style of this project.
    • [ ] My change requires a change to the documentation.
    • [ ] I have updated the documentation accordingly.
    • [x] I have added tests to cover my changes.
    • [x] All new and existing tests passed.
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

Aug 22, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Dec 29, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Nov 3, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Dec 30, 2021
A Go (golang) environment loader (which loads env vars from a .env file)

A Go (golang) environment loader (which loads env vars from a .env file)

Feb 8, 2022
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Nov 12, 2021
Go-config - Config parser for go that supports environment vars and multiple yaml files

go-multiconfig This package is able to parse yaml config files. It supports gett

Jun 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

Apr 11, 2022
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Jan 9, 2023
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Sep 26, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Sep 26, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

Jan 5, 2023
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Jan 2, 2023
Environment variables substitution for Go

envsubst Environment variables substitution for Go. see docs below Installation: From binaries Latest stable envsubst prebuilt binaries for 64-bit Lin

Jan 1, 2023
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

May 11, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

Dec 3, 2021