With the proot support landed, we can do multiarch builds, since --no-scripts is now a requirement.

From the apk side of things, this can be done by simply passing --arch [arch] when setting up the apkdb, e.g. in the InitApkDb step.

I'm told that we can possibly recruit @imjasonh to work on this. Ideally we would probably build images for all of the archs at once, I suppose.

Allowing entrypoint to be blank can be used to force default docker behaviour of running cmd in shell. Also updated examples.

Closes #177.

We could also create a PR to set CMD to /bin/sh -l which would be keep previous behaviour when no command is specified.

apko publish produces a multiarch image index. By default, this includes six common archs, which can be overridden with archs: in the image config YAML, or values in the --arch flag (in increasing order of preference).

An attempt is made to accept both apk-style arch strings ("x86_86") and OCI-style arch strings ("amd64"), but more could probably be done. There's a TODO to support armv7 and armhf (arm/v7 and arm/v6 respectively, in OCI terms).

apk keys are extracted from arch-specific locations (e.g., /usr/share/apk/keys/s390x/*) and placed in the standard location (/etc/apk/keys/*).

apko build and apko build-minirootfs both only produce a single arch's tarball, for the current runtime arch. If the image config specifies archs in this case, a warning is logged to avoid confusion.

While working on GitHub actions integration for a demo, I noticed:

localhost:~/actions/apko-build$ docker run ghcr.io/chainguard-dev/apko:v0.1 
Error: unknown command "/bin/sh" for "apko"
Run 'apko --help' for usage.
2022/02/28 23:03:27 error during command execution: unknown command "/bin/sh" for "apko"

@cpanato @mattmoor what do you think? are we missing something with ko?

Right now apko installs the latest version of each package in the config file. This means that every time a build is done, the latest version of each package is used. Updates happen implicitly.

This approach has a few tradeoffs:

• Fast updates are great for picking up security fixes in a timely manner
• Are simple to operate and do in apko

But they:

• can make post-hoc analysis (SBOMs) more difficult
• introduce another attack vector: a compromised distro has RCE on your system

This issue is to discuss whether the ability to pin versions is useful, and how it should be implemented!

Hand-wavy Idea

Somehow persist each independent package version ~forever. I think SLSA has a recommendation for storage lifecycle.

Use a TUF-stye snapshot role to generate up to date indexes of "latest versions".

Allow specific version pinning in an "apko.lock" style file, while still allowing users to specify just "latest". We shouldn't need any real constraint handling or SAT solving, the two choices are just a specific version or latest.

Have a simple command to update a lock file, and throw that into Dependabot or similar to update very day.

I have no idea what this would look like in apk2 or 3 though so I'm probably missing something.

Other Considerations

I'm not certain whether the alpine repositories persist old versions of packages. We'd need to keep them around somewhere to make this work.

Trying to build the first example mentioned on a mac pro x86 with docker desktop, but it fails:

09:04 $ docker run -v "$PWD":/work distroless.dev/apko build examples/alpine-base.yaml apko-alpine:edge apko-alpine.tar
Aug 25 07:04:02.334 [INFO] loading config file: examples/alpine-base.yaml
Aug 25 07:04:02.342 [INFO] [arch:x86_64] WARNING: ignoring archs in config, only building for current arch (amd64)
Aug 25 07:04:02.342 [INFO] [arch:x86_64] building image 'apko-alpine:edge'
Aug 25 07:04:02.342 [INFO] [arch:x86_64] build context:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   working directory: /tmp/apko-2434288168
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   tarball path:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   use proot: false
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   source date: 1970-01-01 00:00:00 +0000 UTC
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   Docker mediatypes: false
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   SBOM output path: /work
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   arch: x86_64
Aug 25 07:04:02.342 [INFO] [arch:x86_64] image configuration:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   contents:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main]
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     keyring:      []
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     packages:     [alpine-base]
Aug 25 07:04:02.342 [INFO] [arch:x86_64]   entrypoint:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     type:
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     command:     /bin/sh -l
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     service: map[]
Aug 25 07:04:02.342 [INFO] [arch:x86_64]     shell fragment:
Aug 25 07:04:02.342 [INFO] [arch:x86_64] doing pre-flight checks
Aug 25 07:04:02.342 [INFO] [arch:x86_64] building image fileystem in /tmp/apko-2434288168
Aug 25 07:04:02.342 [INFO] [arch:x86_64] initializing apk database
Aug 25 07:04:02.342 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk add --initdb --arch x86_64 --root /tmp/apko-2434288168
Aug 25 07:04:02.346 [INFO] [arch:x86_64] initializing apk world
Aug 25 07:04:02.347 [INFO] [arch:x86_64] initializing apk keyring
Aug 25 07:04:02.347 [INFO] [arch:x86_64] initializing apk repositories
Aug 25 07:04:02.347 [INFO] [arch:x86_64] synchronizing with desired apk world
Aug 25 07:04:02.347 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk fix --root /tmp/apko-2434288168 --no-scripts --no-cache --update-cache --arch x86_64
Error: failed to build layer image: initializing apk: failed to fixate apk world: exit status 1
2022/08/25 07:04:02 error during command execution: failed to build layer image: initializing apk: failed to fixate apk world: exit status 1

apko version

09:06 $ docker run distroless.dev/apko version _ ____ _ __ ___ / \ | _ \ | |/ / / _
/ _ \ | |) | | ' / | | | | / ___ \ | __/ | . \ | || | // _\ || |_|_\ ___/ apko

GitVersion: v0.5.0-92-g1e2e81a GitCommit: 1e2e81a71166ab042cbfdf9b712a861c8751a038 GitTreeState: clean BuildDate: '1970-01-01T00:00:00Z' GoVersion: go1.19 Compiler: gc Platform: linux/amd64

docker version:

09:00 $ docker version Client: Cloud integration: v1.0.28 Version: 20.10.17 API version: 1.41 Go version: go1.17.11 Git commit: 100c701 Built: Mon Jun 6 23:04:45 2022 OS/Arch: darwin/amd64 Context: default Experimental: true

Server: Docker Desktop 4.11.1 (84025) Engine: Version: 20.10.17 API version: 1.41 (minimum version 1.12) Go version: go1.17.11 Git commit: a89b842 Built: Mon Jun 6 23:01:23 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.6 GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 runc: Version: 1.1.2 GitCommit: v1.1.2-0-ga916309 docker-init: Version: 0.19.0 GitCommit: de40ad0

This PR modifies the way we generate the purl data for the apko images, shifting the logic to the options object.

This has the benefit of ensuring we get the same purls in both standards by reusing the same logic.

It also fixes a bug where the purl names were formed incorrectly in the image SBOMs, fixing https://github.com/chainguard-dev/apko/issues/281

New sample image purl:

"purl": "pkg:oci/apko-test@sha256:ea959168730bccc00500a22b58a1891c1b07dc9039273b07d5a833f7fa7d7b54?arch=amd64&os=linux"

/cc @mattmoor @kaniini

Signed-off-by: Adolfo García Veytia (Puerco) [email protected]

Writing apko YAML files can be very repetitive. It would be desirable to be able to have a base configuration that can be inherited from.

Something like:

from: github.com/chainguard-dev/apko/baseimages/alpine-edge.yaml@main

contents:
  packages:
    - nano
    - build-base
    - openssl-dev

I think from should imply a git repository, given the GitOps tooling built around distroless, and kind of feel like go package system. At least, it makes sense to me.

The settings in the config file would be overlaid on top of the base config.

This PR makes image generation deterministic by default:

• timestamps are zeroed out for reproducibility (see https://github.com/chainguard-dev/apko/issues/48)
• apk is writing a tar archive to lib/apk/db/scripts.tar, to support reproducible builds the timestamps inside the archive needs to be deterministic/zeroed.

Note: according to the spec of SOURCE_DATE_EPOCH (https://reproducible-builds.org/specs/source-date-epoch/):

Build processes MUST use this variable for embedded timestamps in place of the "current" date and time.

Since time.Now has been replaced with the zero value (January 1, year 1, 00:00:00.000000000 UTC) there's no need for SOURCE_DATE_EPOCH.

Note: reproducible image generation has been tested with https://diffoscope.org/

Closes https://github.com/chainguard-dev/apko/issues/48

cc @imjasonh

To build/install the task definition (w/ apko) you can run:

KO_DOCKER_REPO=ghcr.io/mattmoor ko apply -Bf config/

This task definition implements the interface defined in github.com/mattmoor/mink to support uploading local source and directing where to publish images (so it can be used with mink apply as well).

You can invoke this task imperatively with the following mink command:

$ mink run task apko --as=apko -- --path=./examples/nginx.yaml

2022/02/19 19:21:58 building image 'gcr.io/mattmoor-chainguard/images' from config file './examples/nginx.yaml'
2022/02/19 19:21:58 build context:
2022/02/19 19:21:58   image configuration: {{[https://dl-cdn.alpinelinux.org/alpine/edge/main] [/etc/apk/keys/[email protected] /etc/apk/keys/[email protected] /etc/apk/keys/[email protected] /etc/apk/keys/[email protected] /etc/apk/keys/[email protected]] [alpine-baselayout nginx]} {service-bundle  map[nginx:/usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"]}}
2022/02/19 19:21:58   working directory: /tmp/apko-2587090319
2022/02/19 19:21:58   tarball path:
2022/02/19 19:21:58 doing pre-flight checks
2022/02/19 19:21:58 building image fileystem in /tmp/apko-2587090319
2022/02/19 19:21:58 initializing apk database
2022/02/19 19:21:58 running: /sbin/apk add --initdb --root /tmp/apko-2587090319
2022/02/19 19:21:58 [apk] OK: 0 MiB in 0 packages
2022/02/19 19:21:58 initializing apk keyring
2022/02/19 19:21:58 installing key /etc/apk/keys/[email protected]
2022/02/19 19:21:58 installing key /etc/apk/keys/[email protected]
2022/02/19 19:21:58 installing key /etc/apk/keys/[email protected]
2022/02/19 19:21:58 installing key /etc/apk/keys/[email protected]
2022/02/19 19:21:58 installing key /etc/apk/keys/[email protected]
2022/02/19 19:21:58 initializing apk repositories
2022/02/19 19:21:58 initializing apk world
2022/02/19 19:21:58 synchronizing with desired apk world
2022/02/19 19:21:58 running: /sbin/apk fix --root /tmp/apko-2587090319 --no-cache --update-cache
2022/02/19 19:21:58 [apk] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
(1/12) Installing musl (1.2.2-r7)
(2/12) Installing busybox (1.35.0-r2)
Executing busybox-1.35.0-r2.post-install
(3/12) Installing alpine-baselayout (3.2.0-r19)
Executing alpine-baselayout-3.2.0-r19.pre-install
Executing alpine-baselayout-3.2.0-r19.post-install
(4/12) Installing libcrypto1.1 (1.1.1m-r2)
(5/12) Installing pcre (8.45-r2)
(6/12) Installing libssl1.1 (1.1.1m-r2)
(7/12) Installing zlib (1.2.11-r3)
(8/12) Installing nginx (1.20.2-r0)
Executing nginx-1.20.2-r0.pre-install
Executing nginx-1.20.2-r0.post-install
(9/12) Installing skalibs (2.11.1.0-r0)
(10/12) Installing s6-ipcserver (2.11.0.1-r0)
(11/12) Installing execline (2.8.2.0-r0)
(12/12) Installing s6 (2.11.0.1-r0)
Executing s6-2.11.0.1-r0.pre-install
Executing busybox-1.35.0-r2.trigger
OK: 8 MiB in 12 packages
2022/02/19 19:21:58 generating supervision tree
2022/02/19 19:21:58 simple service: nginx => /usr/sbin/nginx -c /etc/nginx/nginx.conf -g "daemon off;"
2022/02/19 19:21:58   supervision dir: /tmp/apko-2587090319/sv/nginx
2022/02/19 19:21:58 finished building filesystem in /tmp/apko-2587090319
2022/02/19 19:21:59 built image layer tarball as /tmp/apko-1984638129.tar.gz
2022/02/19 19:21:59 building OCI image 'gcr.io/mattmoor-chainguard/images' from layer '/tmp/apko-1984638129.tar.gz'
2022/02/19 19:21:59 OCI layer digest: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:21:59 OCI layer diffID: sha256:775a08d0e0b33a95dff11e6ffe1511605e9e6ef2646ab39353deeb3344b75577
2022/02/19 19:21:59 output OCI image file to foo.tar.gz

2022/02/19 19:22:02 pushed blob: sha256:3ed57839e782bc6b1c084f25433ef6d92647ec9906070d2ecc48ec31cfa84650
2022/02/19 19:22:02 pushed blob: sha256:5b6a06005c97c31df81455008c3c92078a50c8b07bc394a98c2519f2945c47ff
2022/02/19 19:22:03 gcr.io/mattmoor-chainguard/images: digest: sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e size: 427

gcr.io/mattmoor-chainguard/images@sha256:a2fdd03e2be33056b469fe472917e1c2d19670de04da7963e566cd078771c02e

We have loads of these in our mono repo, and many of these are based on workflows we've forked repeatedly over the years.

I'm tempted to set up chainguard-dev/actions where we can stuff reusable workflows with workflow_call that we can recycle across our various repos over time.

cc @n3wscott @k4leung4

I got this invalid parameter at 'ImageManifest' when pushing to ECR

Error: attaching sboms to amd64 image: writing sbom: All attempts fail:
#1: PUT https://xxx.dkr.ecr.ap-southeast-1.amazonaws.com/v2/platform-images/demo-image/manifests/sha256-186a7f24e5a81f8b56c0c94cc8b343683a93780ca852ad318cf9d2c0385702c1.sbom: UNSUPPORTED: Invalid parameter at 'ImageManifest' failed to satisfy constraint: 'Invalid JSON syntax'

Context: One of apko's core features is that it produces an SBOM for every image it builds. We want these SBOMs to be as complete as possible.

It's becoming possible for apko's SBOM information to be sourced primarily from individual apks, thanks to Melange.

But it's still possible that apko ends up producing an SBOM that's missing information about files in the image. Files not explicitly documented (e.g. via package metadata) are sometimes referred to as "dark files", since downstream consumers have no visibility into why these files are present.

What's needed: Apko should explicitly warn users when it produces an SBOM that doesn't account for all of the files (regular, directory, etc.) in the image.

Problem: Today, when apko builds an image, if none of the packages it adds to the image include an /etc/os-release file, the image will end up with an /etc/os-release file that gets created on-the-fly, which looks like this:

ID=unknown
NAME="apko-generated image"
PRETTY_NAME="apko-generated image"
VERSION_ID=unknown
HOME_URL="https://github.com/chainguard-dev/apko"

This can be problematic. In particular, if someone runs a vulnerability scanner on the image, the scanner will not be able to correctly identify the distro, and thus the scanner won't be able to locate the appropriate advisory/fix data source to use during the scan — which means the scan results will be invalid.

In most cases, apko users probably don't intend to have an unidentified distro like this. It's more likely that they just didn't realize how to add a proper distro /etc/os-release file, which today is done by including a package in the config file that contains an /etc/os-release file.

Therefore, apko should check to see if any package has added an /etc/os-release file and ensure the user sees a warning about the fact that the distro won't be identifiable and how that can lead to problems.

Bumps github/codeql-action from 2.1.36 to 2.1.37.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps actions/checkout from 3.1.0 to 3.2.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.