A tool that allows you to manage Kubernetes manifests for your services in a Git repository

kuberpult Readme for users

About

Kuberpult is a tool that allows you to manage Kubernetes manifests for your services in a Git repository and manage the same version of those services in different environments with differnt configs according to the environment.

kuberpult works best with ArgoCD which applies the manifests to your clusters and kuberpult helps you to manage those manifests in the repository.

kuberpult allows you to lock some services or an entire environment, so automatic deployments (via a typical api call) to those services/environments will be queued until the last lock is removed. Manual deployments (via the UI or a flag in the api) are always possible.

kuberpult does not actually deploy. That part is usually handled by argoCD.

kuberpult has a UI, and it can handle locks. When something is locked, it's version will not be changed. Both environments and microservices can be locked.

Current Version and Queued Version

Every app has a current version on every env (including nil for no version). If a deployment starts while the app/env is locked, instead of changing the current version, the queued_version will be set. When the lock is deleted, the queued version will be deployed.

There is currently no visualization for the queue in the ui, so it can only be seen in the manifest repo as "queued_version" symlink next to "version".

The queue has a length of 0 or 1. Attempting to put a version into the full queue, will overwrite it instead ("last deployment wins").

Release train Overview

What is that?

A release train is a concept that ensures that we deploy often and regularly. The idea is that the train does not wait for you - it will leave (deploy) on time, regardless of how many services/commits are ready.

The train should run often enough to not slow down development, while also giving the testers enough time to look at changes before they go live.

Trigger

The release train needs to be triggered externally - there is nothing in kuberpult that triggers it. The trigger is usually implemented as a jenkins pipeline with a cronjob. See k8s-jenkins-cac.tf in your project.

Environments

There are 2 environments involved:

  • target: this is where the services will be deployed (where the version changes happen).
  • upstream: this is where the system tests are run. It is also the source for the versions of the apps.
Owner
freiheit.com technologies
freiheit.com technologies
Comments
  • SRX-APRASR Provide PR number in Overview Services and GitUrl configuration

    SRX-APRASR Provide PR number in Overview Services and GitUrl configuration

    Added GitUrl variable in frontend config, modified the GetFrontEnd Config to accept the variable, added PR number extraction and modified services to send the the PR number

  • Add link to ArgoCD

    Add link to ArgoCD

    As the state of the cluster can differ from that seen in kuberpult, it could be helpful to have link to ArgoCD for the service on a stage to quickly verify that the cluster actually has the state represented in kuberpult.

  • Warn when manually deployed to production

    Warn when manually deployed to production

    When application is manually deployed to production, check that all its upstream environments are already updated, if not give warning.

    image

    story: SRX-5VUIF9

  • SRX-4JC9OG allow setting the release number when creating new commits

    SRX-4JC9OG allow setting the release number when creating new commits

    This change allows a new parameter for the deploy endpoint version to make the endpoint (nearly) idempotent. The parameter is optional. If it's omitted, kuberpult will use the existing behaviour and use the latest release+1.

    This also slightly changes the http status code. Until now, the status code 204 ("No Content") is used to indicate success. However, we have now two different success outcomes. To reflect this, kuberpult will respond with 201 ("Created") when a new release was created and 200 ("Okay") when the release already exists.

  • Sort releases correctly

    Sort releases correctly

    Allow releases that are created later to be sorted corretly by the commit history (of the source repo).

    Current Situation

    • Main build fails.
    • there is the possibility to retrigger it in jenkins or github actions. However, this will always build the latest version (main branch). Even if there are 2 Commits that are new, these will be lumped together into one kuberpult release. This works technically, but it's confusing.

    Goal:

    • If a main build fails, it can be retriggered with a specific offset (in git commits). Even if there was another successfull main build in between.
    • Jenkins will publish a new version. This new version is then sorted into the right place in kuberpult. It's not necessary the latest release.

    Open Questions:

    • How do we clean up old releases? Do we need to change this?
    • How old can old commits be to be accepted? They shouldn't be older than a few days.
    • When creating an "in-between" release, how do we handle deployment? We need to make sure "upstream" is not deployed then, because it would be a downgrade.
  • Locks improvements

    Locks improvements

    • rename "All Locks" -> "Locks"
    • make 2 secions: "environment locks" and "app locks"
    • in the release dialog, show both the name AND the id of the lock.
  • Feature proposal: Allow release trains per team

    Feature proposal: Allow release trains per team

    We have the use case where kuberpult is used with many teams and each team would like to trigger a release train for their services only. This is especially important if you have quality gateway tests which block the release train:

    e.g. a qualitygateway test of team A currently does block the release train for all services and teams.

    I think it might make sense to allow specifying a prefix for the release train.

    E.g. to say that only services with this prefix should be deployed.

  • SRX-63BV32 If config.json file is invalid, give a fatal error during repository initialization

    SRX-63BV32 If config.json file is invalid, give a fatal error during repository initialization

    https://revolution.dev/app/-JqFGExX46gs9mH7vxR5/-M37cZx7XYoOvQ5Mmcir/FILESYSTEM/FOLDER/ROOT/STORY/-Mp6i5lzSJREYmTNHy6p

    When the config.json file in any of the environments is an invalid json file (missing file is ok) then do not start kuberpult cd-service, throw a valid error message and quit, do the same in /health check as well.

  • Filter serivce by name

    Filter serivce by name

    An easy way to have a filter for large teams, would be to add a query parameter, like ?q=foo this should then only display all services with "foo" in the name.

  • Use buf plugins instead of manually installing protoc-gen-* binaries

    Use buf plugins instead of manually installing protoc-gen-* binaries

    Not having to manually install the protoc-gen-* binaries enables simplifications of the start.sh scripts and makes the docker-compose start more robust. Additionally all tool dependencies required to generate protobuf code is now in buf.gen.yaml instead of split between that file and go.tools.mod.

    The versions used in go.mod and buf.gen.yaml have to be kept in sync! (Previously, the versions used in go.mod and go.tools.mod would have to be kept in sync.)

    This also removes manual installation of libgit2 and a yaml library. libgit2 is already a dependency in go.mod and the yaml library was not used. I included both removals in this commit (even though they are unrelated), because that allowed some simplification of the makefile code in the .install target.

    This also updates protoc-gen-go from 1.26.0 to 1.28.0, protoc-gen-go-grpc from 1.1.0 to 1.2.0, protoc-gen-grpc-gateway from 2.4.0 to 2.10.0.

    Story: SRX-Y9BRXK Story: https://revolution.dev/app/-JqFGExX46gs9mH7vxR5/-M37cZx7XYoOvQ5Mmcir/STORY/-Mwb00UWgoLeu2OucGwW/

  • Undeploy too complicated

    Undeploy too complicated

    Idea: Simplify the process of undeploy. Currently it's a bit long and tedious to wait for all release trains.

    On the first button to create an undeploy version, we could ask the user:

    Do you want to deploy the "undeploy" version immediately? Note that this includes a deployment to every environment. You should only do this if you are sure that the service is not needed anymore on any environment.

    Then add all deployment actions to the cart.

  • Bugfix: Write undeploy versions correctly into manifest repo

    Bugfix: Write undeploy versions correctly into manifest repo

    go-billys WriteFile function has a funny behavior: It does not allow overwriting an existing file with empty content. We work around this by giving the file a single space. For argoCd this should not make a difference

Jenkins CLI allows you manage your Jenkins as an easy way

Quick start 简体中文 Jenkins CLI Jenkins CLI allows you manage your Jenkins in an easy way. No matter if you're a plugin developer, administrator or just

Jan 4, 2023
This is for managing Slack App Manifests, it is no use if you are not developing an App for Slack.

Terraform Provider Slack App This is for managing Slack App Manifests, it is no use if you are not developing an App for Slack. Requirements Terraform

May 23, 2022
Tool for generating Spinnaker application/pipelines and k8s manifests

jarvis Just A Rather Very Intelligent System Get git clone [email protected]:ealebe

Jan 6, 2022
A fluxcd controller for managing remote manifests with kubecfg

kubecfg-operator A fluxcd controller for managing remote manifests with kubecfg This project is in very early stages proof-of-concept. Only latest ima

Nov 1, 2022
crud is a cobra based CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service documentation and k8s deployment manifests

crud crud is a CLI utility which helps in scaffolding a simple go based micro-service along with build scripts, api documentation, micro-service docum

Nov 29, 2021
PolarDB-X Operator is a Kubernetes extension that aims to create and manage PolarDB-X cluster on Kubernetes.

GalaxyKube -- PolarDB-X Operator PolarDB-X Operator is a Kubernetes extension that aims to create and manage PolarDB-X cluster on Kubernetes. It follo

Dec 19, 2022
Crossplane provider to provision and manage Kubernetes objects on (remote) Kubernetes clusters.

provider-kubernetes provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters ty

Jan 3, 2023
Git with a cup of tea, painless self-hosted git service
Git with a cup of tea, painless self-hosted git service

Gitea - Git with a cup of tea View the chinese version of this document Purpose The goal of this project is to make the easiest, fastest, and most pai

Jan 2, 2023
Watchtower for Git: automatically keep local Git repositories up to date with their remotes

CrowsNest Watchtower for Git: automatically keep local Git repositories up to date with their remotes. Configuration Flags --run-once or -r: Normally

Oct 30, 2022
Mattermost outline plugin allows you to search your teams documents.
Mattermost outline plugin allows you to search your teams documents.

mattermost-plugin-outline Mattermost Outline plugin allows you to search your teams documents. Installation In Mattermost 5.16 and later, this plugin

Dec 7, 2022
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.

Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload. Run tools like masscan, puredns, ffuf, httpx or a

Dec 31, 2022
The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your container orchestrator

fortress-csi The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your co

Jan 23, 2022
Open Service Mesh (OSM) is a lightweight, extensible, cloud native service mesh that allows users to uniformly manage, secure, and get out-of-the-box observability features for highly dynamic microservice environments.
Open Service Mesh (OSM) is a lightweight, extensible, cloud native service mesh that allows users to uniformly manage, secure, and get out-of-the-box observability features for highly dynamic microservice environments.

Open Service Mesh (OSM) Open Service Mesh (OSM) is a lightweight, extensible, Cloud Native service mesh that allows users to uniformly manage, secure,

Jan 2, 2023
🐶 Kubernetes CLI To Manage Your Clusters In Style!
🐶 Kubernetes CLI To Manage Your Clusters In Style!

K9s - Kubernetes CLI To Manage Your Clusters In Style! K9s provides a terminal UI to interact with your Kubernetes clusters. The aim of this project i

Jan 9, 2023
ArgoCD is widely used for enabling CD GitOps. ArgoCD internally builds manifest from source data in Git repository, and auto-sync it with target clusters.
ArgoCD is widely used for enabling CD GitOps. ArgoCD internally builds manifest from source data in Git repository, and auto-sync it with target clusters.

ArgoCD Interlace ArgoCD is widely used for enabling CD GitOps. ArgoCD internally builds manifest from source data in Git repository, and auto-sync it

Dec 14, 2022
Synchronise a directory's contents with a git repository.

git-volume-reloader Synchronise a directory's contents with a git repository. Synchronisation is triggered by a webhook sent by the git service provid

Sep 16, 2022
The CLI tool glueing Git, Docker, Helm and Kubernetes with any CI system to implement CI/CD and Giterminism
The CLI tool glueing Git, Docker, Helm and Kubernetes with any CI system to implement CI/CD and Giterminism

___ werf is an Open Source CLI tool written in Go, designed to simplify and speed up the delivery of applications. To use it, you need to describe the

Jan 4, 2023
Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

Nov 10, 2021
Kubernetes OS Server - Kubernetes Extension API server exposing OS configuration like sysctl via Kubernetes API

KOSS is a Extension API Server which exposes OS properties and functionality using Kubernetes API, so it can be accessed using e.g. kubectl. At the moment this is highly experimental and only managing sysctl is supported. To make things actually usable, you must run KOSS binary as root on the machine you will be managing.

May 19, 2021