Environment variables substitution for Go

envsubst

GoDoc License Build status Github All Releases

Environment variables substitution for Go. see docs below

Installation:

From binaries

Latest stable envsubst prebuilt binaries for 64-bit Linux, or Mac OS X are available via Github releases.

Linux and MacOS
curl -L https://github.com/a8m/envsubst/releases/download/v1.2.0/envsubst-`uname -s`-`uname -m` -o envsubst
chmod +x envsubst
sudo mv envsubst /usr/local/bin
Windows

Download the latest prebuilt binary from releases page, or if you have curl installed:

curl -L https://github.com/a8m/envsubst/releases/download/v1.2.0/envsubst.exe
With go

You can install via go get (provided you have installed go):

go get github.com/a8m/envsubst/cmd/envsubst

Using via cli

envsubst < input.tmpl > output.text
echo 'welcome $HOME ${USER:=a8m}' | envsubst
envsubst -help

Imposing restrictions

There are three command line flags with which you can cause the substitution to stop with an error code, should the restriction associated with the flag not be met. This can be handy if you want to avoid creating e.g. configuration files with unset or empty parameters. Setting a -fail-fast flag in conjunction with either no-unset or no-empty or both will result in a faster feedback loop, this can be especially useful when running through a large file or byte array input, otherwise a list of errors is returned.

The flags and their restrictions are:

Option Meaning Type Default
-i input file ```string stdin```
-o output file ```string stdout```
-no-unset fail if a variable is not set flag false
-no-empty fail if a variable is set but empty flag false
-fail-fast fails at first occurence of an error, if -no-empty or -no-unset flags were not specified this is ignored flag false

These flags can be combined to form tighter restrictions.

Using envsubst programmatically ?

You can take a look on _example/main or see the example below.

package main

import (
	"fmt"
	"github.com/a8m/envsubst"
)

func main() {
    input := "welcom $HOME"
    str, err := envsubst.String(input)
    // ...
    buf, err := envsubst.Bytes([]byte(input))
    // ...
    buf, err := envsubst.ReadFile("filename")
}

Docs

api docs here: GoDoc

Expression Meaning
${var} Value of var (same as $var)
${var-$DEFAULT} If var not set, evaluate expression as $DEFAULT
${var:-$DEFAULT} If var not set or is empty, evaluate expression as $DEFAULT
${var=$DEFAULT} If var not set, evaluate expression as $DEFAULT
${var:=$DEFAULT} If var not set or is empty, evaluate expression as $DEFAULT
${var+$OTHER} If var set, evaluate expression as $OTHER, otherwise as empty string
${var:+$OTHER} If var set, evaluate expression as $OTHER, otherwise as empty string
$$var Escape expressions. Result will be $var.

Most of the rows in this table were taken from here

See also

  • os.ExpandEnv(s string) string - only supports $var and ${var} notations

License

MIT

Owner
Comments
  • support for restriction flags -no-unset and -no-empty

    support for restriction flags -no-unset and -no-empty

    I added two flags -no-unset and -no-empty to the envsubst command, with which you can cause the substitution to stop with an error code, should the restriction associated with the flag not be met. This can be handy if you want to avoid creating e.g. configuration files with unset or empty parameters. The flags and their restrictions are:

    |Flag | Meaning | | ------------| -------------- | |-no-unset | fail if a variable is not set |-no-empty | fail if a variable is set but empty

    These flags can be combined to form tighter restrictions.

    Sorry about the fairly large change set...

  • Add actions to test and build binaries for x86_64/arm64

    Add actions to test and build binaries for x86_64/arm64

    Hi!

    We really need to build this binary for arm64 (we're switching to m1 macs) and I figured it wouldn't be fair to ask you to put any work in so I whipped this up. This PR:

    • Tests on Go 1.16/1.17 in github actions
    • Automatically builds linux/darwin x86_64/arm64 binaries whenever you cut a release

    I'd love it if you'd merge this and cut a release :)

  • Feature Request: escape `$$`

    Feature Request: escape `$$`

    Thank you so much for this project, the -no-unset option was exactly what I was looking for, I've immediately replaced our usage of gettext with it.

    Now that I'm a user, would it be possible for envsubst to replace all strings $$foo with $foo?

    In my case, I have kubernetes templates for nginx configuration, which I template with some values, but which also contain nginx variables like $remote_user and $bytes_sent strings. My current choices are either to not use -no-unset, or (what I actually do) implement my own escaping with sed.

  • New flag -d to not replace variables that start with a digit.

    New flag -d to not replace variables that start with a digit.

    This code solves #38. It leaves variables untouched which start with a digit. This is the same behavior that the original gnu envsubst has. This change is fully backward compatible. The new functionality get's only enabled when the flag -d is given on the command line or when the new functions are used programatically. Some examples for variables which would not be replaced anymore are:

    • $1
    • ${1}
    • $2ABC
    • ${2ABC}
  • Replaces $1, $2 etc, which are not environment variables

    Replaces $1, $2 etc, which are not environment variables

    Expressions like $1 get replaced with an empty string, but $1 is by definition no environment variable, because it starts with a digit (See https://stackoverflow.com/a/2821183). This behavior is unexpected different compared to the original gnu envsubst, which does not replace $1.

  • Release binaries by GitHub action

    Release binaries by GitHub action

    Turns out this was an upstream breakage and I had to upgrade the action (they fixed it 5 hours ago!), just terrible timing on the merge and I apologize for you wasting a bunch of time on this. Since the upgrade I can run the action in my fork just fine with no failures, so I think we're good again.

  • feat: add default behaviour of running through all the errors

    feat: add default behaviour of running through all the errors

    #22

    Added a flag to override the defualt - fail-fast will enable users to fail fast if required otherwise all errors are displayed in non relaxed modes

    image

  • Restrictions only restrict when variable inside braces

    Restrictions only restrict when variable inside braces

    If this is intended, apologies... I didn't see it documented.

    package main
    
    import (
    	"log"
    	"github.com/a8m/envsubst"
    )
    
    func main() {
    	process("Hello $name, how are you?")
    	process("Hello ${name}, how are you?")
    }
    
    func process(tpl string) {
    	out, err := envsubst.StringRestricted(tpl, true, true)
    	if err != nil {
    		panic(err)
    	}
    	log.Println("Shouldn't be here - " + out)
    }
    
  • Need a simple way to escape dollar symbols

    Need a simple way to escape dollar symbols

    There needs to be a way to insert dollar symbols in a template Instead of creating a variable (E.G. DOLLAR=\$).

    For instance if envsubst was used in a bash script, replacing all the dollars $ with a variable will be messy.

  • Add support for full error list

    Add support for full error list

    I would like to add support for returning a full list of errors and optionally stop execution on first error. This will be very handy in more complex CI/CD scenarios where config files have a large number of variables to be replaced.

  • Escaping syntax

    Escaping syntax

    Hi,

    nice little helper tool;) Although i recently stumbled over a problem i couldn't find a solution to.. How can i escape sth. like this:

    A=${static}
    

    I know about $${static} -> $static. But what if i need the {} ?

    regards, Roman

  • Add option to only parse $ALL_CAPS env vars

    Add option to only parse $ALL_CAPS env vars

    Especially for nginx configs, lots of directives use $lowercase_variable, so having to escape a lot of content.

    It would be nice to have an caps-only flag to only detect env vars that are uppercase.

  • Feature Request / Possible bug: Support nested variables enclosed by raw string

    Feature Request / Possible bug: Support nested variables enclosed by raw string

    I have a weird issue where envsubst is not working as expected.

    When I add only a single a default variable to the right side of the experssion it will work similarly as a default POSIX shell. But when I add additional text to the default it start to do some strange behaviour.

    I'll provide some examples, I believe that will be the best description.

    sh use case:

    docker run --rm -it debian:buster-slim sh
    # export VAR1="shouldevaluate"
    # export EMPTY_VAR=""
    # export MAIN_VAR="${EMPTY_VAR:-$VAR1}"
    # echo $MAIN_VAR
    shouldevaluate
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe$VAR1}"
    # echo $MAIN_VAR
    asdqweshouldevaluate
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe $VAR1 qweasd}"
    # echo $MAIN_VAR
    asdqwe shouldevaluate qweasd
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe${VAR1}qweasd}"
    # echo $MAIN_VAR
    asdqweshouldevaluateqweasd
    

    And this happens when I try to use the same method in envsubst.

    # cat test.template
    name = ${EMPTY_VAR:-$VAR1}
    
    name = ${EMPTY_VAR:-asdqwe$VAR1}
    
    name = ${EMPTY_VAR:-asdqwe $VAR1 qweasd}
    
    name = ${EMPTY_VAR:-asdqwe${VAR1}qweasd}
    #
    #
    # envsubst < test.template
    name = shouldevaluate
    
    name = asdqwe$VAR1
    
    name = asdqwe $VAR1 qweasd
    
    name = asdqwe${VAR1qweasd}
    
  • Support ignore and keep empty system environments

    Support ignore and keep empty system environments

    e.g I have prometheus config below which want only replace CONSUL_URL from system environment. but it will replace $1 left blank there which make metrics_path not correct.

      - job_name: 'consul'
        metrics_path: '/actuator/prometheus'
        consul_sd_configs:
          - server: '${CONSUL_URL:-127.0.0.1:8500}'
            services: [] #match all service
        relabel_configs:
          - source_labels: [__meta_consul_service]
            target_label: job
          - source_labels: [__meta_consul_service_metadata_management_context_path]
            regex: "(.+)"
            target_label: __metrics_path__
            replacement: '$1/prometheus'
    
  • Add support for env files

    Add support for env files

    I'd suggest the following enhancement of envsubst to be more useful in combination with Docker/Docker Compose or Systemd Unit files:

    • Add option to read environment from env files
    • Allow multiple env files to be given and process them in order (to override values from former files to support development purposes)
    • Add option to ignore the existing shell environment variables and build a fresh environment using the given env file(s) This would greatly simplify some development and deployment processes.
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Jan 9, 2023
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Sep 26, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Nov 3, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Sep 26, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

Jan 5, 2023
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Dec 11, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Dec 26, 2022
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Jan 2, 2023
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

May 11, 2021
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Nov 12, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

Dec 3, 2021
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

Aug 22, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Dec 30, 2021
formicidate is a small tool for Go application can update the value of environment variables in a .env file with code

formicidae Update .env files in Go with code. What is fomicidae? formicidate is a small tool for Go application. You can update the value of environme

Jan 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

Apr 11, 2022
Tmpl - A tool to apply variables from cli, env, JSON/TOML/YAML files to templates

tmpl allows to apply variables from JSON/TOML/YAML files, environment variables or CLI arguments to template files using Golang text/template and functions from the Sprig project.

Nov 14, 2022
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Jan 8, 2023
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Dec 29, 2022