A dead-simple configuration management tool powered by stupid shell scripts.

Efs2

Don't you wish you could configure a server as easily as creating a Docker image? Meet Efs2, A dead simple configuration management tool that is powered by stupid shell scripts.

Efs2 is an idea to combine the stupid shell scripts philosophy of fss with the simplicity of a Dockerfile.

PkgGoDev Go Report Card Build Status Coverage Status

Getting Started

Let's take a look at how easy it is to use Efs2 to configure NGINX on Ubuntu.

Installation

Efs2 is simple to install, with the fastest method being to download one of our binary releases.

It is also possible to install Efs2 with Go (requires v1.14+).

$ go get -u github.com/madflojo/efs2

Once installed, we can start defining our steps to setup NGINX.

The Efs2file

An Efs2file powers Efs2's configuration; much like a Dockerfile, this file uses a simple set of instructions to configure our target servers.

# Install and Configure NGINX

# Run apt-get update
RUN apt-get update --fix-missing && apt-get -y upgrade

# Install nginx
RUN apt-get install nginx

# Deploy Config files
PUT example.com.conf /etc/nginx/sites-available/example.com.conf 0644

# Create a Symlink
RUN ln -s /etc/nginx/sites-available/example.com.conf /etc/nginx/sites-enabled/example.com.conf

# Restart NGINX
RUN systemctl restart nginx

The above Efs2file showcases how simple the Efs2 instructions are. Our NGINX server is configured with two simple instructions RUN and PUT.

The RUN instruction is simple; it executes whatever command you provide. The PUT instruction uploads files. That's it, that's all the instructions included with Efs2. Simple but effective.

Remote Execution

Efs2 uses SSH to execute the instructions specified within the Efs2file. Just run the Efs2 command, followed by the target hosts.

$ efs2 host1.example.com host2.example.com

Command-Line Options

Efs2 offers several additional options, such as parallel execution and various authentication methods.

  -v, --verbose   Enable verbose output
  -q, --quiet     Silence output
  -f, --file=     Specify an alternative Efs2File (default: ./Efs2file)
  -i, --key=      Specify an SSH Private key to use (default: ~/.ssh/id_rsa)
  -p, --parallel  Execute tasks across multiple hosts in parallel
  -d, --dryrun    Print tasks to be executed without actually executing any tasks
      --port=     Define an alternate SSH Port (default: 22)
  -u, --user=     Remote host username (default: current user)
      --passwd    Ask for a password to use for host authentication

Call to Action

Efs2 is a small project to fit the fine line between complex configuration management and simple shell scripts. We are always looking for users to share their stories and contribute to our examples repository.

For those interested in helping develop Efs2. The time, skills, and perspectives you contribute to this project are valued. Please reference our Issues Page for open ideas and our Contributing Guide for contribution details.

If you like Efs2, please tell others about it by sharing this project on the social media site of your choice.

Owner
Benjamin Cane
Thoughts and opinions are my own.
Benjamin Cane
Comments
  • Wee --passwd bug

    Wee --passwd bug

    Describe the bug

    --passwd is part of the CLI. At face value it suggests that it is for the remote account's password, but it is actually for the passphrase of the SSH private key locally.

    Expected behavior

    I would think this would be better as --passphrase or --pki-passphrase

  • put file error with permission deny

    put file error with permission deny

    image

    so, if not root user, how to offer authentication info to efs2 so that it can copy the file to root permission folders?

    run sudo is not an option, I think

  • why don't make RUN CMD and RUN SCRIPT just be RUN?

    why don't make RUN CMD and RUN SCRIPT just be RUN?

    say, the following works:

    RUN CMD cd elasticsearch-6.2.3 && sudo sysctl -w vm.max_map_count=262144

    but this will NOT:

    RUN CMD cd elasticsearch-6.2.3 && ./bin/elasticsearch -d -p pid or RUN CMD cd elasticsearch-6.2.3 && ./bin/elasticsearch -d -p pid

    but it's a common sense to combine and run like this.

    we can parse this line and send to server, I don't think this is difficult, but I am not a gopher so I can't make my hand dirty, hope this can be done in the near future.

    :)

  • Document RUN SCRIPT

    Document RUN SCRIPT

    Describe the idea

    After I discovered efs2 shortly I wondered how one could execute arbitrary scripts or programs remotely. Sure you, can

    PUT script.sh /tmp/script.sh 0700 RUN /tmp/script.sh; rm /tmp/script.sh

    But that seemed not very userfriendly to me.

    Then I discovered in that there were once RUN CMD and RUN SCRIPT commands. Now there is only RUN (which is the same as RUN CMD I think). RUN SCRIPT is still there, but undocumented.

    Describe possible implementations

    Keep RUN SCRIPT and document it.

    Suggested Improvement: For security reasons mktemp(1) should be used to create the temporary script on the target.

  • Recipes repo - efs2-recipes

    Recipes repo - efs2-recipes

    Goal

    An exchange for people wanting to read and contribute to Efs2file scripts that purport to do something useful that others may want to copy.

    Design

    Some directory structure:

    components/
      nginx/
        static-content/
          Efs2file
          readme.md
      sinatra/
        hello-world/
          Efs2file
          readme.md
    applications/
      plex-server
        default-setup/
          Efs2file
          readme.md
    

    Examples

    The Nginx example of the main README could also exist in this directory structure.

    Unknowns

    Efs2 doesn't purport to provide a language that adapts to apk/apt/rpm differences, so that might need to be codified in the file or directory naming:

    applications/
      plex-server
        default-setup-debian8-and-lower/
          Efs2file
          readme.md
        default-setup-debian9-and-above/
          Efs2file
          readme.md
    
  • Fixes #19 and adds some pop to the documentation

    Fixes #19 and adds some pop to the documentation

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    This change adds some fine-tuning to the documentation but it also addresses the issue of undocumented legacy instructions being available within Efs2.

    Type of change

    Please delete options that are not relevant.

    • [X] New feature (non-breaking change which adds functionality)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Multiline Shell commands are not supported

    Multiline Shell commands are not supported

    Describe the bug

    Currently, only single-line shell commands are supported.

    To Reproduce

    This works:

    RUN this command does stuff
    

    This doesn't:

    RUN this command \
    does stuff
    

    Expected behavior

    When using multi-line commands, Efs2 should detect the \ and adjust the commands appropriately.

  • Updated README

    Updated README

    Description

    Small README change

    Type of change

    Please delete options that are not relevant.

    • [X] This change requires a documentation update

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Adding Password Authentication

    Adding Password Authentication

    Description

    This pull request adds the ability to use the --passwd flag to force Efs2 to ask the user for a password. The password will then be used for authentication to the remote SSH server.

    Type of change

    Please delete options that are not relevant.

    • [X] New feature (non-breaking change which adds functionality)
    • [X] This change requires a documentation update

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Adding a quiet flag

    Adding a quiet flag

    Description

    This change adds a command line and configuration flag to silence execution output. It does not change any exit codes as users will still rely on those for understanding success or failure.

    Type of change

    Please delete options that are not relevant.

    • [X] New feature (non-breaking change which adds functionality)
    • [X] This change requires a documentation update

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Adding Passphrase support for SSH Keys

    Adding Passphrase support for SSH Keys

    This will fix #1 by adding an error handler after a failed attempt at parsing an encrypted private key. The error handler will prompt the user for the passphrase and try again.

  • Default config file

    Default config file

    Describe the idea

    It would be an advantage to have a configuration file where you could write the default in, so you don't have to always give the parameters.

    Describe possible implementations

    In the HOME directory, look in the .config folder for an efs2 with a file named config.yml. The structure could be as follows:

    verbose: false
    ssh_key: ~/.ssh/id_dsa
    ssh_port: 2000
    ssh_user: root
    

    Maybe it is possible to adjust config/config.go so that it first searches for the config file, and then reads the parameters of the input from the terminal.

  • Strategy for linux-dialog style interactive shells (popular with OS installers on raspberry-pi sized devices)

    Strategy for linux-dialog style interactive shells (popular with OS installers on raspberry-pi sized devices)

    Describe the idea

    I've a fleet os Raspberry Pi's and smaller less well known devices (Odroid). First boot or first SSH connection of these things oftent goes into an interactive session for initial setup. Completing that workflow before attaching Efs2 for proper post boot setup is a need.

    Describe possible implementations

    Dialog is a thing that people use. Automating it is a frequent question on Stack Overflow (etc) - https://superuser.com/questions/1041735/automating-dialog - one of the answers links to https://stackoverflow.com/questions/3804577/have-bash-script-answer-interactive-prompts. This seems limited though.

    There's https://pypi.org/project/webssh/ which could be used with Selenium (I'm co-creator of v1 back in 2004) to do any timing sensitive navigation of DOM representation of the terminal, and complete that setup.

    With that, you'd launch wssh in the background, open selenlium-webdriver, do the needed one-time setup, exit, kill the wssh process.

    Then you'd go back in with Efs2 to o the rest (reset default passwords, packages, etc)

    Question: Is this the right approach or does Efs2 have a different way?

    Additional context

    Some little devices have something closer to cloud-init for setup, but there's huge variance for the different OSs that can be used.

  • Brew install

    Brew install

    Describe the idea

    The installation of Efs2 should be as easy as possible. For Mac OS, that means getting into the Brew list of installable programs.

    Describe possible implementations

    Brew has custom taps that can be used to package and distribute Efs2.

    https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

    Additional context

    Probably need another issue like this for Linux.

  • Multiline string way of specifying PUT

    Multiline string way of specifying PUT

    Describe the idea

    PUT command enhanced like so for new optional way:

    PUT """ blah
            blah
            blah ...""" /etc/nginx/nginx.conf 0644
    

    Additional context

    In some cases items to be PUT on the target vm are short/clear and inlining them might boost the quick learning of the Es2file.

  • efs2 does not accept encrypted private keys

    efs2 does not accept encrypted private keys

    With passphrase on key:

    efs2 -i /Users/hbokh/.ssh/id_rsa -v -u root hostname.home
    Unable to setup SSH client configuration - Unable to read private key file - ssh: cannot decode encrypted private keys
    

    With empty passphrase on key (host was down hence the other errors):

    efs2 -i /Users/hbokh/.ssh/id_rsa_nopassphrase -v -u root hostname.home
    linux008.internal:22: Executing task 0 - RUN CMD apt-get update
    linux008.internal:22: Error connecting to host - dial tcp 192.168.1.1:22: getsockopt: operation timed out
    linux008.internal:22: Error executing command - dial tcp 192.168.1.1:22: getsockopt: operation timed out
    

    Since I only use passphrased keys, this is somewhat of a showstopper for me... Can this be fixed?

A dead simple, no frills Go cross compile tool

Gox - Simple Go Cross Compilation Gox is a simple, no-frills tool for Go cross compilation that behaves a lot like standard go build. Gox will paralle

Jan 1, 2023
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.

Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.

Jan 8, 2023
Lightweight and dead-simple CI detection.

is-ci Lightweight and dead-simple CI detection for golang. This mod is based on the @npmcli/ci-detect package. Install go get -u github.com/wesleimp/i

Dec 17, 2021
crud is a cobra based CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service documentation and k8s deployment manifests

crud crud is a CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service docum

Nov 29, 2021
Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd data, and intelligent diagnosis.
Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd data, and intelligent diagnosis.

Kstone 中文 Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd

Dec 27, 2022
Go version manager. Super simple tool to install and manage Go versions. Install go without root. Gobrew doesn't require shell rehash.

gobrew Go version manager Install or update With curl $ curl -sLk https://git.io/gobrew | sh - or with go $ go get -u github.com/kevincobain2000/gobre

Jan 5, 2023
Terraform utility provider for constructing bash scripts that use data from a Terraform module

Terraform Bash Provider This is a Terraform utility provider which aims to robustly generate Bash scripts which refer to data that originated in Terra

Sep 6, 2022
preflight helps you verify scripts and executables to mitigate chain of supply attacks such as the recent Codecov hack.
preflight helps you verify scripts and executables to mitigate chain of supply attacks such as the recent Codecov hack.

?? Mitigate chain of supply attacks ?? Verify your curl scripts and executables ?? What is it? preflight helps you verify scripts and executables to m

Nov 18, 2022
Use this program to embed sh scripts in binaries

sh2bin Use this program to embed sh scripts in binaries. Feel free to fork this

Jan 4, 2023
Linux provisioning scripts + application deployment tools. Suitable for self-hosting and hobby-scale application deployments.

Apollo Linux provisioning scripts + application deployment tools. Suitable for self-hosting and hobby-scale application deployments. Philosophy Linux-

Feb 7, 2022
cloudquery powered by Osquery

cloudquery powered by Osquery cloudquery is Osquery extension to fetch cloud telemetry from AWS, GCP, and Azure. It is extensible so that one can add

Dec 25, 2022
TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative.

TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative. TriggerMesh allows you to declaratively define event flows between sources and targets as well as add even filter, splitting and processing using functions.

Dec 30, 2022
API for global CO2 measurements, powered by the Ribbit Network

Ribbit Network API (WIP) This repository contains the backend services supportin

Dec 22, 2022
Planet Scale Robotics - Offload computation-heavy robotic operations to GPU powered world's first cloud-native robotics platform.

robolaunch ?? Planet Scale Robotics - Offload computation-heavy robotic operations to GPU powered world's first cloud-native robotics platform. robola

Jan 1, 2023
Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.

Packer Website: https://www.packer.io IRC: #packer-tool on Freenode Mailing list: Google Groups Packer is a tool for building identical machine images

Jan 8, 2023
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster

Kube-Knark Project Trace your kubernetes runtime !! Kube-Knark is an open source tracer uses pcap & ebpf technology to perform runtime tracing on a de

Sep 19, 2022
Tool for creating identical machine images for multiple platforms from a single source configuration.

Packer Packer is a tool for building identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs o

Jan 18, 2022
A tool to bring existing Azure resources under Terraform's management

Azure Terrafy A tool to bring your existing Azure resources under the management of Terraform. Install go install github.com/magodo/aztfy@latest Usage

Dec 9, 2021