A build tool from space, down on earth.

Bob

Inspired by Make and Bazel · Made for humans

Latest Release GoDoc Build Status

Bob is a build system, a task runner as well as tooling for Git Multi-repos, all bundled into a single binary.

With bob, you can:

  • Build your programs efficiently, as Bob tracks build inputs and caches compiled outputs, providing fast incremental builds.
  • Run local development environments, whether they are simple binaries, docker-compose files, or a mix of both.
  • Multi-Repo Tooling Easily manage multi-repo setups, with bulk Git operations.

⚠️ ⚠️ ⚠️ Warning: Bob and its documentation is in a early stage. Most of the features are fully-functional and tested, but some details are yet to be ironed out.

Install

To install and start using Bob, do:

go install github.com/Benchkram/bob

For shell autocompletion (bash and zsh supported) add source <(bob completion) to your .bashrc/.zshrc.

Build System

Bobs generates its internal build graph from tasks described in a bob.yaml file (we usually refer to it as "bobfile").
The basic components of a task are:

  • input: Whenever an input changes, the task's commands need to be re-executed [default: *]
  • cmd: Commands to be executed
  • target: File(s) or directories that are created when the commands are run and can be reused in other tasks.

Example of a bob.yaml file:

tasks:
  build:
    input: ./main.go
    cmd: go build -o ./app
    target: ./app

Multiline sh and bash commands are entirely possible, and are powered by mvdan/sh.

Take a look into the server-db example for a step-by-step tutorial.

Local Development

Our goal is to create a world-class local development experience by integrating seamlessly with the build-system and enabling you to start right after cloning a repository. Let the build-system create binaries and docker images, then execute & control them from one terminal using bob run. No more back-and-forth tab switching.

Ideally, those are the only two commands necessary to get started when joining a new project that uses Bob:

git clone
bob run

Web Server Example

Individual steps for web server development are likely similar to this:

  1. Code generation using an IDL, like openapi or protobuf
  2. Compile a server binary
  3. Run a database in docker
  4. Run the server

Those build/run tasks can be described in a bob.yaml file. This allows bob run to launch and give you control to a local development environment.

An in-depth example is available here.

Multi-repo Git Tooling

Bob enables a natural feeling git workflow for Git multi-repo setups without relying on Git Submodules.

To do this, Bob uses the concept of a "workspace" to track other git repositories. Inside a workspace you can use the usual every day git-operations through bob with visually enhanced output for the multi-repo cases.

Here is an example of bob git status in a workspace:

Highlighted directories indicate child Git repositories and therefore bob git add and bob git commit will only operate on the repository a file belongs to allowing to create repository overlapping commits all together.

⚠️ So far only bob clone and bob git status are implemented.

Take a look into the git package's README for the current status of bob git

Setting Up a Workspace

To set up a new bob workspace you first have to initialize it:

bob workspace init

This creates a .bob.workspace file and initializes a new git repository in the current directory.

Adding repositories to the workspace:

bob workspace add [email protected]:Benchkram/bob.git
bob clone # calls git clone for missing repos

Cloning an existing workspace from a remote git repository:

bob clone [email protected]:Benchkram/bob.git

Dependencies

A list of Bob's top dependencies:

  • charmbracelet/bubbletea - Used in Bob's Terminal User Interface
  • docker/compose - Enables us to run docker-compose files the same way docker compose does
  • sh - Parsing/execution of shell commands
Owner
benchkram
Software Company From Stuttgart, Germany
benchkram
Comments
  • Adding native libraries as dependencies

    Adding native libraries as dependencies

    I have a rust project with serde as a dependency. When I tried to use bob to build it, I got the following error

    = note: ld: library not found for -liconv
            clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
    

    I tried to add libiconv to the dependencies in bob.yaml but the error remained

    dependencies:
      - cargo
      - clang_13
      - libiconv
    ...
    

    However, I could compile it successfully with nix-shell -p cargo clang_13 libiconv.

    How do I add these native libraries so that bob can find them?

  • install script

    install script

    Testing

    Download the latest version

    You can omit cd if you don't want the binary installed globally.

    cd /usr/local/bin
    curl -L https://raw.githubusercontent.com/benchkram/bob/b71503511064242db6c05f637491e16cd1f0d881/install.sh | sh
    

    Download a specific version

    You can omit cd if you don't want the binary installed globally.

    cd /usr/local/bin
    curl -L https://raw.githubusercontent.com/benchkram/bob/b71503511064242db6c05f637491e16cd1f0d881/install.sh | BOB_VERSION=0.5.1 sh
    

    To test the file locally

    Checkout this branch and:

     env BOB_VERSION=0.5.2 ./install.sh
    

    then you can run:

    ./bob --version
    

    You can also specify BIN_DIR which will install bob in working-directory/BIN_DIR/

     env BOB_VERSION=0.5.2  BIN_DIR=bin ./install.sh
    
  • Configurable project name

    Configurable project name

    Closes #12

    • If empty, set project name based on the project's absolute filesystem path
    • Validate uniqueness of project name across aggregated bobfiles
    • Project name passed to tasks, so that artifacts can be properly stored if needed (e.g. in a remote store)
    • Playground generation slightly modified to help test the project name
  • bob init

    bob init

    closes #265

    Usage:

    In a directory without bob.yaml:

    bob init for projects without remote cache bob init projectURL with remote cache

    When running bob build afterwards don't forget about '--insecure'

  • bob init command

    bob init command

    Maybe we should create a bob init which should similar to git init.

    bob init creates a bob.yaml if it doesn't exist in the working directory with a build command which does an echo hello world. Maybe the nixpkgs can be pinned to latest stable version

    bob init projectURL should work similiar to bob init but also add a project.

    If there is already a bob.yaml file inside working directory it should display a user error.

  • Full hermetic mode for run tasks

    Full hermetic mode for run tasks

    Once https://github.com/benchkram/bob/issues/97 is merged we should add https://github.com/benchkram/bob/issues/131 functionality for run tasks as well.

  • Remove artifact store interface

    Remove artifact store interface

    We can remove Store interface

    Following reasons:

    1. Clean() is implemented only on the local, Done only on the remote
    2. ctx was added to the interface only for the remote store purpose
    3. In the code they are always used separately so no polymorphism is happening.
    4. existence of "not implemented" methods on both implementations

    1 & 2 show they really are different behaviours so no benefit of having this interface. Adding new functionalities to one implementation forces us to add params to the second one.

  • Bob run: Correct error message if blocking docker-containers are running

    Bob run: Correct error message if blocking docker-containers are running

    ###Scenario Have a docker container running which is using e.g. port 8080. Have a run task which starts a container which also would use port 8080.

    ###Current behavior Bob will just exit with either no message or a bad stack trace.

    ###Expected behavior Proper user error parsed info from docker-compose message.

  • `bob run` shows output only after initial build

    `bob run` shows output only after initial build

    This blocks the TUI for a too long time

    The commander should return without starting any long running tasks. Upstream code can then start the build after the TUI is started.

    https://github.com/benchkram/bob/blob/36981b6f3c49f839e5678550a358f6faa8473daf/bob/run.go#L52

  • Bump github.com/spf13/viper from 1.13.0 to 1.14.0

    Bump github.com/spf13/viper from 1.13.0 to 1.14.0

    Bumps github.com/spf13/viper from 1.13.0 to 1.14.0.

    Release notes

    Sourced from github.com/spf13/viper's releases.

    v1.14.0

    What's Changed

    Enhancements 🚀

    Breaking Changes 🛠

    Dependency Updates ⬆️

    Full Changelog: https://github.com/spf13/viper/compare/v1.13.0...v1.14.0

    Commits
    • b89e554 chore: update crypt
    • db9f89a chore: disable watch on appengine
    • 4b8d148 refactor: use new Has fsnotify method for event matching
    • 2e99a57 refactor: rename watch file to unsupported
    • dcb7f30 feat: fix compilation for all platforms unsupported by fsnotify
    • 2e04739 ci: drop dedicated wasm build
    • b2234f2 ci: add build for aix
    • 52009d3 feat: disable watcher on aix
    • b274f63 build(deps): bump github.com/fsnotify/fsnotify from 1.5.4 to 1.6.0
    • 7c62cfd build(deps): bump github.com/stretchr/testify from 1.8.0 to 1.8.1
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • Bump github.com/deepmap/oapi-codegen from 1.10.1 to 1.12.4

    Bump github.com/deepmap/oapi-codegen from 1.10.1 to 1.12.4

    Bumps github.com/deepmap/oapi-codegen from 1.10.1 to 1.12.4.

    Release notes

    Sourced from github.com/deepmap/oapi-codegen's releases.

    Ensure Scopes context key are generated as valid Go names

    As highlighted by @​KenxinKun in deepmap/oapi-codegen#874, gorilla/mux (as well as some other servers) were affected by an invalid Scopes context key.

    This makes them valid Go names.

    Fix the generated error handling for Gin

    In v1.12.0 we added support for custom error handling to be used in the generated Gin code.

    Unfortunately, we didn't produce the right code, so this would cause (compilation) errors.

    Fix big issue with marshaling binary strings

    v1.12.0 introduced a File type which intercepts schemas of type:string, format:binary, but it incorrectly implemented the json.Marshaler interface and was lacking tests to catch the problem. This version fixes the marshaling of fields of that type.

    Prior versions of oapi-codegen turned those into a []byte, and you can still annotate your spec with x-go-type if you need []byte

    Fix an issue with command line parsing

    • The -o flag specifying the output filename was being ignore in very simple invocations of oapi-codegen without a configuration file. This is fallout from trying to be compatible with old and new style configs.

    Many bug fixes

    Major changes to functionality

    • In the last release, we changed the configuration file format to a new syntax, and added flags to read the old version, but it turns out that this default-on change broke a lot of automation, sorry about that. In this release, we've done our best to auto-detect the configuration version, so both styles of config options should work. The explicit version selection flags have not been removed.
    • Gorilla router (deepmap/oapi-codegen#594)
    • "strict" server generation, meaning one which is much more pedantic about input and output types, which allows for making API clients and servers that look more like Go functions than HTTP handlers (deepmap/oapi-codegen#499)
    • Fields can be marked with x-json-ignore to omit in JSON serialization (deepmap/oapi-codegen#390)
    • Many fixes to unions for oneOf and anyOf handling, thank you committers.
    • Enums can be prefixed with their typename in all cases, versus only on collisions previously. This is a configurable option (deepmap/oapi-codegen#662)
    • The Client interface in generated code was named awfully generically. You can now override the name (deepmap/oapi-codegen#788)
    • Generate code can be optionally formatted using more initialisms. It's default-off to not break existing code (deepmap/oapi-codegen#749)
    • Gin code can be configured with a custom error handler, versus generating error responses inline. Default behavior is the same as before (deepmap/oapi-codegen#587)
    • Gin middleware evaluation order can be reversed to be consistent with common Gin usage, it's default-off to not break previous behavior. ( deepmap/oapi-codegen#787)

    Everything that has changed

    This is a very large release with many bug fixes. Thanks to all contributors.

    ... (truncated)

    Commits
    • a444d30 Merge pull request #881 from KenxinKun/fix-unsanitised-template
    • 55b0905 Fixes missing unsanitised templates
    • 5a33791 Merge pull request #877 from KenxinKun/gorilla-fix-context-scope-variable-tem...
    • d14917d Fixes missing sanitation of context key variable in gorilla server
    • 4ec8015 Merge pull request #854 from deepmap/dependabot/go_modules/golang.org/x/tools...
    • 9b992be Gitignore bin
    • fe8408c Bump golang.org/x/tools from 0.2.0 to 0.3.0
    • 40890b2 Merge pull request #722 from jamietanna/chore/golangci-lint
    • 0138e13 Fix error handler for required query parameters (#857)
    • 5a103c5 Add linting with golangci-lint
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • `target dirs` avoid deleting user generated target dirs

    `target dirs` avoid deleting user generated target dirs

    In case of directory targets

    task:
      targets:
        dir/
    

    bob can add a file called .bobgenerated to indicate this directory is in control of bob.

    This allows to detect that a directory or any file inside this directory is safe to delete for Bob.

  • Hotrealoading

    Hotrealoading

    closes https://github.com/benchkram/bob/issues/240 and https://github.com/benchkram/bob/issues/291

    On rebuild we now check file per file If there is a target node_modules, the node_modules/.cache dir will be ignored Make sure to bob clean system before testing

    Remaining todos:

    • [x] Investigate and fix the only failing test
    • [x] Remove file content hash on artifact meta if is not necessary(mandatory for benchmarks)
    • [x] Check if bob clean targets still works
    • [x] Check failing CI
    • [x] Double check code for other cleanups

    For the issue when the symflink from targets changes, this is partially fixed, currently it detects the size has changed. For a proper solution we need to store the file reference in buildinfo.

    Steps to reproduce:

      sym:
        cmd: |-
          touch hellox
          ln -s hellox shortcut
        target: |
          hellox
          shortcut
    

    Dp a build, ls -la to validate. Then change the symlink pointer to a different file: ln -sfn anotherFile shortcut. Run build again. The build should be cached and shortcut point back to hellox.

  • Incorrect artifact creation when target is a directory starting with dot

    Incorrect artifact creation when target is a directory starting with dot

    A task with directory target starting with . will remove the dot when creating the artifact.

     targetwithdirs:
        cmd: |-
          mkdir -p .bbuild/dirone/dirtwo
          touch .bbuild/dirone/fileone
          touch .bbuild/dirone/filetwo
          touch .bbuild/dirone/dirtwo/fileone
          touch .bbuild/dirone/dirtwo/filetwo
        target: .bbuild/dirone/
    

    The above task will create an artifact with bbuild instead of .bbuild breaking the subsequent unpacks.

Automatically power off system when network interface is down

passer A tiny tool can automatically power off system when network interface is

Apr 23, 2022
Automated-gke-cilium-networkpolicy-demo - Quickly provision and tear down a GKE cluster with Cilium enabled for working with Network Policy.

Automated GKE Network Policy Demo Before running the automation, make sure you have the correct variables in env-automation/group_vars/all.yaml. There

Jan 1, 2022
Azure Kubernetes Service (AKS) advanced networking (CNI) address space calculator.

aksip Azure Kubernetes Service (AKS) advanced networking (CNI) address space calculator. Download Download the the latest version from the releases pa

Dec 23, 2022
a build tool for Go, with a focus on cross-compiling, packaging and deployment

goxc NOTE: goxc has long been in maintenance mode. Ever since Go1.5 supported simple cross-compilation, this tool lost much of its value. There are st

Dec 9, 2022
A tool to build, deploy, and release any environment using System Containers.
A tool to build, deploy, and release any environment using System Containers.

Bravetools Bravetools is an end-to-end System Container management utility. Bravetools makes it easy to configure, build, and deploy reproducible envi

Dec 14, 2022
A tool to build, deploy, and release any application on any platform.
A tool to build, deploy, and release any application on any platform.

Waypoint Website: https://www.waypointproject.io Tutorials: HashiCorp Learn Forum: Discuss Waypoint allows developers to define their application buil

Dec 28, 2022
Build powerful pipelines in any programming language.
Build powerful pipelines in any programming language.

Gaia is an open source automation platform which makes it easy and fun to build powerful pipelines in any programming language. Based on HashiCorp's g

Jan 3, 2023
Build Go Toolchains /w native libs for cross-compilation

gonative Cross compiled Go binaries are not suitable for production applications because code in the standard library relies on Cgo for DNS resolution

Dec 20, 2022
"go build" wrapper to add version info to Golang applications

govvv The simple Go binary versioning tool that wraps the go build command. Stop worrying about -ldflags and go get github.com/ahmetb/govvv now. Build

Dec 16, 2022
Multi cluster kubernetes dashboard with batteries included. Build by developers, for developers.

kubetower Multi cluster kubernetes dashboard with batteries included. Built by developers, for developers. Features Restart deployments with one click

Nov 28, 2022
Build and deploy Go applications on Kubernetes
Build and deploy Go applications on Kubernetes

ko: Easy Go Containers ko is a simple, fast container image builder for Go applications. It's ideal for use cases where your image contains a single G

Jan 5, 2023
"go build" wrapper to add version info to Golang applications

govvv The simple Go binary versioning tool that wraps the go build command. Stop worrying about -ldflags and go get github.com/ahmetb/govvv now. Build

Dec 16, 2022
A simple Kubernetes Operator template that uses Golang, use it to build your own operators
A simple Kubernetes Operator template that uses Golang, use it to build your own operators

A simple programmatic Kubernetes Operator template. Use this to create your own Kubernetes operators with golang. Build with KIND (Kubernetes in Docke

May 13, 2022
Build a retractable ECS load balance network through aliyun openapi.

ECSEquilizer 通过阿里云OpenAPI建立一个可伸缩的负载均衡网络调度器。 简介 为确保代理集群网络和计算能力可以通过ECS云服务动态伸缩,所以制定实现以下策略。 节点分为static和dynamic两种,static节点是通过配置文件(config.yaml)实现预设的,固定不变。 d

Jul 2, 2022
How to build production-level services in Go leveraging the power of Kubernetes

Ultimate Service Copyright 2018, 2019, 2020, 2021, Ardan Labs [email protected] Ultimate Service 3.0 Classes This class teaches how to build producti

Oct 22, 2021
Boxygen is a container as code framework that allows you to build container images from code

Boxygen is a container as code framework that allows you to build container images from code, allowing integration of container image builds into other tooling such as servers or CLI tooling.

Dec 13, 2021
The example shows how to build a simple multi-tier web application using Kubernetes and Docker
The example shows how to build a simple multi-tier web application using Kubernetes and Docker

Guestbook Example This example shows how to build a simple multi-tier web application using Kubernetes and Docker. The application consists of a web f

Nov 15, 2021
A simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app to Docker Hub

go-pipeline-demo A repository containing a simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app t

Nov 17, 2021
crud is a cobra based CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service documentation and k8s deployment manifests

crud crud is a CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service docum

Nov 29, 2021