The Monogon Project Monorepo. May contain traces of peanuts and a pure Go Linux userland.

Monogon Monorepo

This is the main repository containing the source code for the Monogon Project.

⚠️ This is pre-release software that happens to be publicly available. Nothing to see here, please move along.

Environment

Our build environment requires a working Podman binary (your distribution should have one).

Usage

Spinning up: scripts/create_container.sh

Spinning down: scripts/destroy_container.sh

Running commands: scripts/run_in_container.sh <...>

Using bazel using a wrapper script: scripts/bin/bazel <...> (add to your local $PATH for convenience)

IntelliJ

This repository is compatible with the IntelliJ Bazel plugin, which enables full autocompletion for external dependencies and generated code. All commands run inside the container, and necessary paths are mapped into the container.

The following steps are necessary:

  • Install Google's Bazel plugin in IntelliJ. On IntelliJ 2020.3 or later, you need to install a beta release of the plugin.

  • Add the absolute path to your ~/.cache/bazel-nxt folder to your idea64.vmoptions (Help → Edit Custom VM Options) and restart IntelliJ:

    -Dbazel.bep.path=/home/leopold/.cache/bazel-nxt

  • Set "Bazel Binary Location" in Other Settings → Bazel Settings to the absolute path of scripts/bin/bazel. This is a wrapper that will execute Bazel inside the container.

  • Use File → Import Bazel project... to create a new project from .bazelproject.

After running the first sync, everything should now resolve in the IDE, including generated code.

It's strongly recommend to use our project presets for file watchers and other IDE features. Run this command and re-open the project in order to install them:

bazel run intellij/localconfig $(pwd)

Metropolis

Run a single node cluster

Launch the node:

scripts/bin/bazel run //:launch

Run a kubectl command:

scripts/bin/bazel run //metropolis/cli/dbg -- kubectl describe

Run tests:

scripts/bin/bazel test //...
Comments
  • JDK necessary in builder image?

    JDK necessary in builder image?

    Seeing "Installing adwaita-cursor-theme-3.36.1-1.fc32.noarch" scroll past as create_container.sh was running, I looked into where that dependency came from. It's java-11-openjdk:

    https://github.com/monogon-dev/monogon/blob/6ef7f9bb94890748cc7c635f187fce7c5f497fe3/build/ci/Dockerfile#L34

    I then removed this package from build/ci/Dockerfile and re-ran create_container.sh, scripts/bin/bazel clean && scripts/bin/bazel test //... and tried the IntelliJ plugin, and everything appeared to continue to work as expected.

    My recommendation would be:

    • If the included JDK is indeed unnecessary, it can be removed -- no need to include it and its dependencies in the builder image!
    • If the JDK is necessary, it seems likely that java-11-openjdk-headless would suffice. This would avoid a few themes, fonts and Xorg dependencies from being pulled in, which are almost certainly unused.
  • Implement cluster lifecycle management

    Implement cluster lifecycle management

    (originally created by @q3k in T933)

    Implementation of the cluster lifecycle part of our lifecycle design doc:

    • node/cluster state machine
    • multi-node clusters (replace golden ticket mechanism)
  • Shipping Firmware, Microcode and other blobs

    Shipping Firmware, Microcode and other blobs

    We will need network/storage firmware for some platforms sooner or later.

    We should also almost definitely ship Intel Microcode (and other CPU vendor?) updates.

    Both of these fall under a generic umbrella of 'shipping blobs', be it from linux-firmware or elsewhere. One important thing to investigate is whatever licensing issues might arise from this, perhaps look into how some Linux Distribution Vendors do it.

  • Curator Leader Failover

    Curator Leader Failover

    Currently, the Curator cannot function in follower mode, and the roleserver only dials the first node of the cluster (ie. the curator that should be the leader, until failover).

    We should implement this and exercise this in some end-to-end test.

  • Supervisor TestPanic flake

    Supervisor TestPanic flake

    Observed on https://jenkins.monogon.dev/job/gerrit-presubmit-monogon/job/44%252F444%252F1/1/console

    --- FAIL: TestPanic (0.05s)
        supervisor_test.go:372: runnable 'one' didn't acknowledge cancel
    FAIL
    
  • Escrow Kubernetes Credentials

    Escrow Kubernetes Credentials

    We currently don't have a way to get Kubernetes credentials without using the Debug Service.

    We should add an RPC endpoint that peforms yet another credential escrow: from cluster manager/client credentials to Kubernetes credentials.

    Or, we should get kube-apiserver to accept Metropolis credentials? I think we don't want to do that thought, because the apiserver has no CRL functionality... To investigate.

  • gRPC logging/tracing middleware

    gRPC logging/tracing middleware

    We currently have no way to log anything in the Curator's gRPC handlers. This isn't great.

    We should use the opportunity to introduce this properly, eg. integrate with some tracing system, or at least make it possible to easily integrate with such a tracing system later.

    Some design decisions to be made:

    • Do we make supervisor.Logger(ctx) just work? How supervisor-agnostic do we want to make this? We likely want to use logtree interfaces, but maybe not refer to the supervisor package unless necessary? Is the clean separation worth having to do supervisor.Logger in runnables vs. somethingelse.Logger in RPCs?
    • Where do we actually log things from requests? Somewhere within the logtree of the supervisor runnable, or to a totally separate set of buffers?
  • Bump versions before MVP release

    Bump versions before MVP release

    What we should bump:

    • [X] Go (to 1.17),
    • [X] gVisor (to a recent release)
    • [x] Linux (to newest release within LTS, or new LTS)
    • [x] Kubernetes (to 1.22/1.23)
    • [x] Fedora Container (to 35)

    We we should consider:

    • [x] Other Go dependencies?
  • Storage: system partition integrity

    Storage: system partition integrity

    Metropolis needs to check whether that the rootfs (a.k.a. system partition) mounted by the kernel hasn't been tampered with, at least according to the built/shipped kernel. For that, we need to put more work into how we build images.

    The image build flow currently is:

    • //metropolis/node:rootfs generates an erofs image containing all node userland code.
    • //third_party/linux builds a kernel with EFI stub that hardcodes its boot command line to console=ttyS0 root=PARTLABEL=METROPOLIS-SYSTEM rootfstype=erofs init=/init
    • //metropolis/node:image takes the above rootfs and EFI image and combines them into a disk image with an EFI system partition, the erofs image as its own partition, and an empty data partition for Metropolis to use. This is used by the //metropolis/test/launch code to actually run Metropolis, alongside a config protobuf.

    We should generally uncouple these things, as currently we have some unwritten expectations about how things should work, but they effectively only work for our test code.

    First, we need some way to enforce integrity on erofs system partition images (“Checksum” them). For example, using dm-integrity or dm-verity. I don't know how these works, but this should likely checksum an erofs image (perhaps adding some supporting structures to it) and yield the resulting image and an accompanying checksum (probably in some small protobuf file).

    Then, we need to have a way to “Lock” a Kernel that it should expect an erofs with such-and-such integrity guarantees (ie. a given checksum). This should be done without having to rebuild the kernel. @lorenz knows of some UEFI/Linux feature that should allow us to do this by adding a command line to an existing EFI payload. We shouldn't end up with something that itself isn't further checkable, so keeping to a single EFI image is IMO the best option.

    Finally, the kernel and erofs images must be “Joined” into a pair where a we get an EFI binary and erofs image that will work together. This can then be piped again into //metropolis/node:image (which perhaps should live in //metropolis/test/launch)?

    I think these features should be part of the build process, and not an end-user tool, as users are unlikely to want to dynamically build such images outside of the build system - instead, they will either use a release from Monogon or do their own build if they want to run a patched/forked version of Metropolis. Perhaps later we will make up some sort of artifact format (ie. tarball :) ) that contains both of these together as a single file for redistribution, but for now just a Bazel target with two outputs should be good enough.

    Or, graphically:

       .---------------------------.          .---------------------------.
       | erofs image (no checksum) |          | kernel image (no cmdline) |
       |  //metropolis/node:rootfs |          |    //third_party/linux    |
       '---------------------------'          '---------------------------'
                   |                                      |
                   | “Checksum”                           |  
                   :----------------------.               |
                   |                      |               |
                   V                      V               |
       .-------------------------.  .----------------.    |
       | erofs image  (checksum) |  | checksum proto |    |
       '-------------------------'  '----------------'    |
                   |                      | .-------------'
                   |                      | | “Lock”
                   |                      V V
                   |              .---------------------------.
                   |              | kernel image (expects     |
                   |              | erofs image with checksum |
                   |              '---------------------------'
                   |                      |
                   | .--------------------'
                   | | “Join”
                   V V
          .--------------------------------------.
          | joined kernel EFI and erofs image    |
          | //metropolis/node:joined.{efi,img} ? |
          '--------------------------------------'
    
         
    

    Note: I came up with all these names right now - these are not written in stone, and neither is this design. Whoever implements this should probably first write a short design document for this :).

    Not in scope: further signing this joined pair into some sort of self-standing release.

    Not in scope: building an installer or anything that is 'ready to use'.

  • Node EFI image doesn't boot on some real hardware

    Node EFI image doesn't boot on some real hardware

    @msgctl and @lorenz have been doing some troubleshooting of Metropolis on real hardware.

    The symptoms are that after the firmware boots into the image, we get a blank screen. This might be our EFI stub, this might be the Linux EFI stub we then launch, it might be some Linux configuration. It probably doesn't come back after calling ExitBootServices.

    @lorenz has just got some hardware to debug this (LPC bus sniffing for outb() tombstone debugging)

  • Race between reaper and Go process management

    Race between reaper and Go process management

    (originally reported by @lorenz in T968)

    Hit during tests (<<0.1% probability):

    supervisor E0412 12:24:11.249739 supervisor_processor.go:198] Runnable root.enrolment died: returned error when NODE_STATE_NEW: could not make and mount data partition: formatting encrypted block device: wait: no child processes

    Likely caused by a race condition between pid1 reaping and Go's command.Run().

Related tags
BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go
BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go

BadgerDB BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the underlying database for Dgraph, a fast,

Dec 10, 2021
Eagle - Eagle is a fast and strongly encrypted key-value store written in pure Golang.

EagleDB EagleDB is a fast and simple key-value store written in Golang. It has been designed for handling an exaggerated read/write workload, which su

Dec 10, 2022
Low-level key/value store in pure Go.
Low-level key/value store in pure Go.

Description Package slowpoke is a simple key/value store written using Go's standard library only. Keys are stored in memory (with persistence), value

Jan 2, 2023
A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

go-mysql-server is a SQL engine which parses standard SQL (based on MySQL syntax) and executes queries on data sources of your choice. A simple in-memory database and table implementation are provided, and you can query any data source you want by implementing a few interfaces.

Dec 27, 2022
Pure Go implementation of D. J. Bernstein's cdb constant database library.

Pure Go implementation of D. J. Bernstein's cdb constant database library.

Oct 19, 2022
pure golang key database support key have value. 非常高效实用的键值数据库。
pure golang key database support key have value.  非常高效实用的键值数据库。

orderfile32 pure golang key database support key have value The orderfile32 is standard alone fast key value database. It have two version. one is thi

Apr 30, 2022
A lightweight document-oriented NoSQL database written in pure Golang.
A lightweight document-oriented NoSQL database written in pure Golang.

Lightweight document-oriented NoSQL Database ???? English | ???? 简体中文 | ???? Spanish CloverDB is a lightweight NoSQL database designed for being simpl

Jan 1, 2023
🤔 A minimize Time Series Database, written from scratch as a learning project.
🤔 A minimize Time Series Database, written from scratch as a learning project.

mandodb ?? A minimize Time Series Database, written from scratch as a learning project. 时序数据库(TSDB: Time Series Database)大多数时候都是为了满足监控场景的需求,这里先介绍两个概念:

Jan 3, 2023
Simple DB using yaml. A project for managing the content of yaml files.

DB Yaml Simple DB using yaml. A project for managing the content of yaml files. Table of Contents DB Yaml Features Usage Write to DB Query DB Get Firs

Dec 27, 2022
Toy project to test golang toolset, using as pretext the "cyclic number" problem (famous 142857 number)

go-cyclic-number Toy project to test golang toolset, using as pretext the "cyclic number" problem (famous 142857 number) First version: https://github

Feb 12, 2022
Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on commands and key-regexes written by GO

Welcome to NIPO Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on command

Dec 28, 2022
Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures.

Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures. capabilities which owl provides include Process approval、sql Audit、sql execute and execute as crontab、data backup and recover .

Nov 9, 2022
Being played at The Coffee House and try to find and play it on Spotify
Being played at The Coffee House and try to find and play it on Spotify

The Coffee House Muzik Follow the music that is being played at The Coffee House and try to find and play it on Spotify. Installation Clone this proje

May 25, 2022
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.

Walrus is a fast, secure and reliable backup system suitable for modern infrastructure. With walrus, you can backup services like MySQL, PostgreSQL, Redis, etcd or a complete directory with a short interval and low overhead. It supports AWS S3, digitalocean spaces and any S3-compatible object storage service.

Jan 5, 2023
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

Sep 26, 2022
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support

BuntDB is a low-level, in-memory, key/value store in pure Go. It persists to disk, is ACID compliant, and uses locking for multiple readers and a sing

Dec 30, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Dec 31, 2022
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache Summary groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many case

Dec 29, 2022
Scalable datastore for metrics, events, and real-time analytics

InfluxDB InfluxDB is an open source time series platform. This includes APIs for storing and querying data, processing it in the background for ETL or

Jan 5, 2023