I found this issue working with Hermit in VS Code. When we launch VS Code from a hermit-activated environment/shell, it fails to set the PATH correctly for the underlying terminal:

source ./bin/activate-hermit
echo $HERMIT_ENV                       # /path/to/my/repo
which python3                          # /path/to/my/repo/bin/python3

Which tells me that the environment is activated successfully, however, if I open a VS Code instance from that terminal:

code /path/to/my/repo

And try to use the integrated terminal:

echo $HERMIT_ENV                       # /path/to/my/repo
which python3                          # /usr/bin/python3

Which means that HERMIT_ENV is propagated, but PATH is not propagated correctly. To fix this, I have to activate it again from the integrated terminal, even though it is using zsh, which has the right hermit shell-hooks in .zshrc:

source ./bin/activate-hermit

This Hermit environment has already been activated. Skipping

I then get this confusing message, however, the shell is configured correctly now:

which python3                          # /path/to/my/repo/bin/python3

I wonder if there is a better way to detect this better than using HERMIT_ENV. We can either:

1. fix this for integrated terminals, so that PATH is propagated correctly.
2. Detect that the shell is not truly configured, and update error message accourdingly.

commands.go was looking increasingly large. I broke it into separate files and it's own package to keep things tidy.

Note that some small commands (noop, version) I left behind just because they were small and more... operational).

Also a caveat is that app/commands/ does not have access to Config, so I had to work around that by binding the http client and the env vars. I'm open to better ideas there..

Added a little something to the makefile, in case it eventually gets bigger. Looks like this: )

Populate sha256sums in a specified manifest file Usage: hermit manifest add-digests <manifest.hcl>

It should populate(in place) all the sha256sums for all the version/platform combinations in the original manifest file.

Testing:

• An integration test has been added to verify the digest created.
• An integration test has been added to verify that hermit install fails on wrong digest.

on a mac with zsh you need gmake ( not make ) , so that source command works in your makefiles.

maybe worth adding this to https://cashapp.github.io/hermit/usage/get-started/#initialising-a-project

then the init works:

	# add to zsh so the hermit hooks can fire.
	hermit shell-hooks --zsh

	# create a local bin with the hermit stuff.
	hermit init

	# loads the env.
	source ./bin/activate-hermit

hermit install python3-3.9.11
pip3 install poetry==1.2.0b1
poetry install

Poetry appears to be having a problem with all packages with the cache permissions.

Installing collected packages: setuptools
    Attempting uninstall: setuptools
      Found existing installation: setuptools 60.9.3
      Uninstalling setuptools-60.9.3:
  ERROR: Could not install packages due to an OSError: Cannot move the non-empty directory '/Users/damien/Library/Caches/hermit/pkg/python3-3.9.11/install/lib/python3.9/site-packages/_distutils_hack/': Lacking write permission to '/Users/damien/Library/Caches/hermit/pkg/python3-3.9.11/install/lib/python3.9/site-packages/_distutils_hack/'.

• [x] 📈 Improve the comments
• [x] ❌ Remove the special case in exec_cmd.go
• [x] 🏁 Check for bin/hermit instead of bin/hermit.hcl
• [x] ❓ Move bin/hermit check into env

We verify the downloaded install script's SHA-256 value against a known checksum in bin/hermit. This ensures the integrity of the installer when bootstrapping hermit.

We version (via file name suffix) the install script by the script's short hash value, so that an install script can be uniquely identified by its version / file name. The corresponding bin/hermit is generated with hermit init for a project.

Note that an install script's version" is unrelated to the version of the hermit executable.

• We make changes for the generated bin/hermit to correctly identity the download URL for the canary channel. This part was overlooked in an earlier implementation.
• The InstallScriptSHAs map is manually populated for the "canary" and "stable" channels, respectively, in env.go.
• Update ScriptSHAs for the updated bin/hermit scripts, for "canary" and "stable" channels.
• Add InstallerSHA256Sums map to the Config struct of the main Hermit app.
• Update GHA CI step for sanity check of SHA256 sums of install script.
• Fix a buggy integration test.

When I use hermit's shellhooks in zsh, in a hermit enabled repo, I get the following error at the end of every single command I run.

fatal:hermit: /Users/yunchi/Projects/bork/bin/hermit has an unknown SHA256 signature (9413f2347c5f70e6a004e62b7faac99d3bb1666f86451ed1f6e05a679e3bc27c); verify that you trust this environment and run 'hermit init /Users/yunchi/Projects/bork'

This doesn't happen if I disable the shell hook and activate hermit manually.

If the hermit DB is deleted, executing binaries from any existing packages would result in an ETag check on each invocation.

I think this behaviour is okay, but it's also not clear to me why the original behaviour was desirable?

Proposed Solution (2.b.) from https://github.com/cashapp/hermit/issues/310.

Example Usage

Without additional sources:

example_config=$(cat <<EOF
description = "Description for example"
binaries = ["example"]
channel "unstable" {
 update = "5m"
 source = "[email protected]:cashapp/example.git#$EXAMPLE_VERSION"
}
EOF
)

override_sources=$(cat <<EOF
{"example": $(echo "$example_config" | jq -R -s '.')}
EOF
)

override_uri="data:application/json;base64,$(echo "$override_sources" | base64)"

HERMIT_ADDITIONAL_SOURCES="$override_uri" hermit ...

Example URI

> echo "$override_uri"
data:application/json;base64,eyJleGFtcGxlIjogImRlc2NyaXB0aW9uID0gXCJEZXNjcmlwdGlvbiBmb3IgZXhhbXBsZVwiCmJpbmFyaWVzID0gW1wiZXhhbXBsZVwiXQpjaGFubmVsIFwidW5zdGFibGVcIiB7CiB1cGRhdGUgPSBcIjVtXCIKIHNvdXJjZSA9IFwiZ2l0QGdpdGh1Yi5jb206Y2FzaGFwcC9leGFtcGxlLmdpdCNtYWluXCIKfQoifQo

I expect this is a little off field but will give it a try

https://docs.nats.io/nats-concepts/jetstream/obj_store

why ?

• Store the binaries inside nats. so no github repo bloat.
• leafs nodes are the perfect analogy to a offline setup.
• AAA security is backed in with JWT and highly extensible.
• scales infinity and is clustered.

The reasons i thought of this is because NAST is a pub sub model, and Hermit is wanted to subscribe to change to the upstream repos. SO its really a similar thing. NATS can subscribe to releases from upstreams and then see the release, and pull it and consumes can have it magically be available for their Hermit systems and developers.

Follow up from #342

The current Renovate plugin supports updating versions of the binaries, but not the version of the hermit binary itself.

This is usually specified in the bin/hermit file, and it defaults to using the stable channel:

export HERMIT_DIST_URL="${HERMIT_DIST_URL:-https://github.com/cashapp/hermit/releases/download/stable}"

This means projects using Hermit can and will be broken if a bad version of Hermit gets released to Stable channel.

One solution would be to set a custom env var HERMIT_DIST_URL like https://github.com/cashapp/hermit/releases/download/v0.31.1. But that has two problems:

1. This variable needs to be set before Hermit is actually invoked and loaded, which means that we need an external way to set it, other than the supported/recommended way in hermit.hcl env property.
2. That will quickly go out of sync, and users need to update it manually regularly.

I suggest that if a repo locks the default installation URL in bin/hermit to specific versions, that Renovate bot updates it during regular updates. This means:

1. Users will stay up to date.
2. No need for an external (additional to Hermit) solution to manage environment variables before Hermit gets loaded.
3. This change is backwards compatible. users can still stay on stable channel if they want, and the bot won't change it if so.

Follow up from https://github.com/cashapp/hermit/issues/342

Users can use a specific git reference to fix their package sources in hermit.hcl sources property. Example:

# hermit.hcl
sources = [
  "https://github.com/cashapp/hermit-packages.git#ref"
]

This means it will quickly get out of date, and it needs updating regularly.

I suggest adding support for this in the Renovate bot, by updating the ref suffix part of the URL regularly: https://docs.renovatebot.com/modules/manager/hermit/

Note: for most use cases, this means fetching the latest commit from the default branch of the repository they are using (origin/HEAD), but for completeness, it can support specifying a custom remote reference (like refs/HEAD/branch-name) for users to put into their renovate.json config, for repositories that use a separate release branch.

This was in the documentation but not implemented (as discovered in #342). #<tag> was supported for Git packages, so I've factored that code out and reused it.

Added an integration test for this, along with a couple of new test helper functions that should make these kind of tests simpler in the future.

A recent update in hermit-packages broke all rust pipelines: https://github.com/cashapp/hermit-packages/issues/235

To avoid this, projects typically fix their dependencies versions. As Hermit docs suggest, this can be done by locking the version of the packages sources they import.

An optional # suffix can be added to checkout a specific tag.

Which suggests something like this:

# hermit.hcl
sources = [
  "https://github.com/cashapp/hermit-packages.git#tag"
]

However, I had the following issues using that feature:

One: hermit-packages does not push tags regularly

For users to stay up to date, there need to be regular tags/versions pushed from this repository. Looking at the existing tags, I find only one index tag that is 6 months old: https://github.com/cashapp/hermit-packages/tags

If pushing tags more regularly is useful for Hermit long-term plans, should it be automated? Otherwise, I suggest supporting any ref in the URL, so that users can specify a specific commit hash (instead of a tag), and update it regularly.

Two: url format is not clear

Now I tried using the following formats, which all failed:

• https://github.com/cashapp/hermit-packages.git#index
  • fatal:hermit: /github/workspace/bin/hermit.hcl: unsupported source
• https://github.com/cashapp/hermit-packages#index.git
  • exit status 128: git sync failed: git clone --depth=1 failed

I suggest adding an explicit example of a versioned url in sources array in the documentation, for clarity.

Three: regular updates

Looking at Renovate plugin docs, it is not clear to me if the bot can update tags (or commit hashes) in the sources URLs in hermit.hcl .. Is this a supported scenario?

This forces every package that is about to be cached to have a digest to check against.

This is a building block for enforcing digest checks if we want to.