/hold for module fixups ref: https://github.com/kubernetes/org/issues/3106

FYI to previous contributors of this code: @puerco @hasheddan @saschagrunert @mattmoor @cpanato @hectorj2f @jayonlau

(Also, woooo, I found yet another way to migrate content!)

Figure out what to migrate:

❯ go list -deps ./cmd/bom/... | grep k8s.io/release/
k8s.io/release/pkg/license
k8s.io/release/pkg/provenance
k8s.io/release/pkg/spdx
k8s.io/release/cmd/bom/cmd
k8s.io/release/cmd/bom

Package up the commit history as packages:

❯ git format-patch -o /tmp/mergepatchs \
$(git log cmd/bom/ docs/bom/ pkg/license/ pkg/provenance/ pkg/spdx/|grep ^commit|tail -1|awk '{print $2}')^..HEAD \
cmd/bom/ docs/bom/ pkg/license/ pkg/provenance/ pkg/spdx/

/tmp/mergepatchs/0001-k8s.io-release-pkg-license-package.patch
/tmp/mergepatchs/0002-License-package-implementation-fakes.patch
/tmp/mergepatchs/0003-License-package-tests.patch
/tmp/mergepatchs/0004-Use-dateless-license-boilerplate-in-all-generated-go.patch
/tmp/mergepatchs/0005-Update-import-references-to-sigs.k8s.io-release-util.patch
/tmp/mergepatchs/0006-Move-from-ioutil-to-os-io-packages.patch
/tmp/mergepatchs/0007-Spin-SPDX-into-own-package.patch
/tmp/mergepatchs/0008-Refactor-SPDX-pkg-to-implementation.patch
/tmp/mergepatchs/0009-SPDX-Document-builder.patch
/tmp/mergepatchs/0010-Image-analyzer-packages.patch
/tmp/mergepatchs/0011-Split-license-interface-to-catalog-obj.patch
/tmp/mergepatchs/0012-Split-license-logic-to-implementation.patch
/tmp/mergepatchs/0013-Update-license-and-download-code-to-new-interfaces.patch
/tmp/mergepatchs/0014-Write-license-files-in-prallel-and-unexport-catalog-.patch
/tmp/mergepatchs/0015-Update-license-tests-and-fakes.patch
/tmp/mergepatchs/0016-SPDX-pkg-unit-and-integration-tests.patch
/tmp/mergepatchs/0017-Regenerate-SPDX-package-fakes.patch
/tmp/mergepatchs/0018-BOM-utility-Generate-SPDX-docs.patch
/tmp/mergepatchs/0019-Consolidate-license-directories-and-naming.patch
/tmp/mergepatchs/0020-Fix-bug-where-lic-dnld-errors-where-lost.patch
/tmp/mergepatchs/0021-Go-module-supprt.patch
/tmp/mergepatchs/0022-Directory-scanning-support.patch
/tmp/mergepatchs/0023-Full-support-for-.gitignore-files-and-patterns.patch
/tmp/mergepatchs/0024-Add-package-dependency-capabilities.patch
/tmp/mergepatchs/0025-Import-add-go-packages-as-pkg-deps.patch
/tmp/mergepatchs/0026-Ensure-package-files-are-checksummed-at-the-right-ti.patch
/tmp/mergepatchs/0027-Scan-go.mod-when-checking-directories.patch
/tmp/mergepatchs/0028-Correctly-implement-LicenseInfoFromFiles-in-spdx-pac.patch
/tmp/mergepatchs/0029-Handle-go-module-downloads-by-commit-sha.patch
/tmp/mergepatchs/0030-Allow-bom-generate-command-to-read-directories-from-.patch
/tmp/mergepatchs/0031-Parallelize-go-module-downloads.patch
/tmp/mergepatchs/0032-Shift-license-output-to-debug-and-fix-div-0-bug.patch
/tmp/mergepatchs/0033-Include-full-list-of-transient-dependencies-in-SBOM.patch
/tmp/mergepatchs/0034-Regenerate-SPDX-fakes.patch
/tmp/mergepatchs/0035-Move-spdx-keywords-NONE-and-NOASSERTION-to-constants.patch
/tmp/mergepatchs/0036-Prewarm-license-cache-during-release.PrepareWorkspac.patch
/tmp/mergepatchs/0037-Reduce-SPDX-file-verbosity.patch
/tmp/mergepatchs/0038-HOWTO-Guide-about-creating-a-BOM.patch
/tmp/mergepatchs/0039-bom-utility-README-file.patch
/tmp/mergepatchs/0040-bom-command-documentation.patch
/tmp/mergepatchs/0041-Add-support-for-configuring-BOM-in-a-YAML-file.patch
/tmp/mergepatchs/0042-Add-more-fields-to-document-build-options.patch
/tmp/mergepatchs/0043-Unit-test-file-for-doc-builder.patch
/tmp/mergepatchs/0044-Parallelize-file-processing.patch
/tmp/mergepatchs/0045-Fix-error-in-go-modules-flag.patch
/tmp/mergepatchs/0046-Make-directory-paths-relative.patch
/tmp/mergepatchs/0047-Use-cached-package-replacements.patch
/tmp/mergepatchs/0048-Avoid-erasing-modules-from-GOPATH.patch
/tmp/mergepatchs/0049-Correct-cache-data-directories-handling.patch
/tmp/mergepatchs/0050-Return-full-license-text-after-scanning.patch
/tmp/mergepatchs/0051-Switch-to-license.ReadTopLicense.patch
/tmp/mergepatchs/0052-Regenerate-license-fakes.patch
/tmp/mergepatchs/0053-Use-native-errors-methods-instead-of-fmt.Sprintf.patch
/tmp/mergepatchs/0054-spdx.buildIDString-function.patch
/tmp/mergepatchs/0055-Fix-spdx-date-bug.patch
/tmp/mergepatchs/0056-Download-all-platforms-when-generating-spdx-packages.patch
/tmp/mergepatchs/0057-Object-interface.patch
/tmp/mergepatchs/0058-SPDX-Relationships.patch
/tmp/mergepatchs/0059-Fix-template-bug.patch
/tmp/mergepatchs/0060-Fix-bug-downloading-packages.patch
/tmp/mergepatchs/0061-Add-ExternalDocRef-to-SPDX-package.patch
/tmp/mergepatchs/0062-Regenrate-layer-SPDX-ID-to-add-imageref.patch
/tmp/mergepatchs/0063-SBOM-Link-source-SBOM-to-artifacts-SBOM.patch
/tmp/mergepatchs/0064-Ensure-SPDX-relationship-rednering.patch
/tmp/mergepatchs/0065-SPDX-Support-relationships-with-remote-entities.patch
/tmp/mergepatchs/0066-License-Catalog-Ignore-deprecated-license-IDs.patch
/tmp/mergepatchs/0067-SPDX-Correctly-form-layer-ref-for-single-img-ref.patch
/tmp/mergepatchs/0068-fix-panic-when-LICENSE-file-is-not-found.patch
/tmp/mergepatchs/0069-Clean-up-extra-spaces.patch
/tmp/mergepatchs/0070-lint-revive-Fixup-variable-declaration-warnings.patch
/tmp/mergepatchs/0071-lint-nolintlint-Enforce-stricter-settings.patch
/tmp/mergepatchs/0072-Fix-SPDX-relationship-order-btw-images-and-ref.patch
/tmp/mergepatchs/0073-Support-license-definition-from-command-flag.patch
/tmp/mergepatchs/0074-go-pkg-provenance.patch
/tmp/mergepatchs/0075-generate-provenance-fakes.patch
/tmp/mergepatchs/0076-pkg-provenance-tests.patch
/tmp/mergepatchs/0077-Add-possibility-to-build-custom-Kubernetes-forks.patch
/tmp/mergepatchs/0078-provenance-Add-LoadStatement-func.patch
/tmp/mergepatchs/0079-provenance-Add-Test-data.patch
/tmp/mergepatchs/0080-provenance-Add-ClonePredicate-function.patch
/tmp/mergepatchs/0081-provenance-statment.VerifySubjects.patch
/tmp/mergepatchs/0082-VerifySubjects-unit-test.patch
/tmp/mergepatchs/0083-lint-gosec-Add-nolint-s-for-SHA1-file-integrity-usag.patch
/tmp/mergepatchs/0084-lint-gosec-Guard-against-decompression-bombs-when-us.patch
/tmp/mergepatchs/0085-lint-gosec-Guard-against-decompression-bombs-when-us.patch
/tmp/mergepatchs/0086-spdx-Fix-panic-when-unable-to-dnload-err.patch
/tmp/mergepatchs/0087-SPDX-Parser.patch
/tmp/mergepatchs/0088-SPDX-Banner.patch
/tmp/mergepatchs/0089-lint-Fix-typo.patch
/tmp/mergepatchs/0090-spdx-Document-outliner.patch
/tmp/mergepatchs/0091-bom-document-outline-subcommand.patch
/tmp/mergepatchs/0092-Switch-to-buffered-output-of-the-outline.patch
/tmp/mergepatchs/0093-spdx-Use-1-to-represent-unlimited-recursion-in-outli.patch
/tmp/mergepatchs/0094-spdx-Make-regular-expressions-global-to-avoid-recomp.patch
/tmp/mergepatchs/0095-spdx-parser-Fix-bug-in-File.Name-and-NONE-Dnld.patch
/tmp/mergepatchs/0096-spdx-SLSA-Provenance-export.patch
/tmp/mergepatchs/0097-Regenerate-provenance-fakeks.patch
/tmp/mergepatchs/0098-provenance-Rewrite-links-to-artifact-locations-in-th.patch
/tmp/mergepatchs/0099-Fix-several-typos-nits-in-comments.patch

Apply the patches to the new repo:

❯ git am /tmp/mergepatchs/*.patch

Applying: k8s.io/release/pkg/license package
Applying: License package implementation fakes
Applying: License package tests
Applying: Use dateless license boilerplate in all generated go files
Applying: Update import references to sigs.k8s.io/release-utils
Applying: Move from ioutil to os/io packages
Applying: Spin SPDX into own package
Applying: Refactor SPDX pkg to implementation
Applying: SPDX Document builder
Applying: Image analyzer packages
Applying: Split license interface to catalog obj
Applying: Split license logic to implementation
Applying: Update license and download code to new interfaces
Applying: Write license files in prallel and unexport catalog options
Applying: Update license tests and fakes
Applying: SPDX pkg unit and integration tests
Applying: Regenerate SPDX package fakes
Applying: BOM utility: Generate SPDX docs
.git/rebase-apply/patch:217: trailing whitespace.
the libraries developed for the Kubernetes SBOM for other
warning: 1 line adds whitespace errors.
Applying: Consolidate license directories and naming
Applying: Fix bug where lic dnld errors where lost
Applying: Go module supprt
Applying: Directory scanning support
Applying: Full support for .gitignore files and patterns
Applying: Add package dependency capabilities
Applying: Import add go packages as pkg deps
Applying: Ensure package files are checksummed at the right times
Applying: Scan go.mod when checking directories
Applying: Correctly implement LicenseInfoFromFiles in spdx packages
Applying: Handle go module downloads by commit sha
Applying: Allow bom generate command to read directories from arguments
Applying: Parallelize go module downloads
Applying: Shift license output to debug and fix div/0 bug
Applying: Include full list of transient dependencies in SBOM
Applying: Regenerate SPDX fakes
Applying: Move spdx keywords NONE and NOASSERTION to constants
Applying: Prewarm license cache during release.PrepareWorkspaceStage()
Applying: Reduce SPDX file verbosity
Applying: HOWTO Guide about creating a BOM
.git/rebase-apply/patch:35: trailing whitespace.
To generate a Bill of Materials for your project, download `bom`, our utility
.git/rebase-apply/patch:40: trailing whitespace.
bom -n 'http://mybom.com/' .
.git/rebase-apply/patch:44: trailing whitespace.
for more information about our tools, SBOMs, and the SPDX standard.
.git/rebase-apply/patch:64: trailing whitespace.
SBOM can express all of them in the same document.
.git/rebase-apply/patch:90: trailing whitespace.
by the SPDX Workgroup, a project of the Linux Foundation.
warning: squelched 6 whitespace errors
warning: 11 lines add whitespace errors.
Applying: bom utility README file
.git/rebase-apply/patch:16: trailing whitespace.
Bill of Materials project. It enables software authors to generate an
.git/rebase-apply/patch:23: trailing whitespace.
has a built-in license classifier that recognizes the 400+ licenses in
.git/rebase-apply/patch:32: trailing whitespace.
have created a
.git/rebase-apply/patch:74: trailing whitespace.
bom generate -n http://example.com/ --image k8s.gcr.io/kube-apiserver:v1.21.0
.git/rebase-apply/patch:86: trailing whitespace.
    -f other/file.txt
warning: 5 lines add whitespace errors.
Applying: bom command documentation
.git/rebase-apply/patch:459: trailing whitespace.
Create SPDX compliant Bill of Materials
.git/rebase-apply/patch:465: trailing whitespace.
## Summary
.git/rebase-apply/patch:474: trailing whitespace.
the libraries developed for the Kubernetes SBOM for other
.git/rebase-apply/patch:523: trailing whitespace.
Create SPDX compliant Bill of Materials
.git/rebase-apply/patch:529: trailing whitespace.
## Summary
warning: 5 lines add whitespace errors.
Applying: Add support for configuring BOM in a YAML file
Applying: Add more fields to document build options
Applying: Unit test file for doc builder
Applying: Parallelize file processing
Applying: Fix error in go modules flag
Applying: Make directory paths relative
Applying: Use cached package replacements
Applying: Avoid erasing modules from GOPATH
Applying: Correct cache/data directories handling
Applying: Return full license text after scanning
Applying: Switch to license.ReadTopLicense()
Applying: Regenerate license fakes
Applying: Use native `errors` methods instead of `fmt.Sprintf`
Applying: spdx.buildIDString function
Applying: Fix spdx date bug
Applying: Download all platforms when generating spdx packages from images
Applying: Object interface
Applying: SPDX Relationships
Applying: Fix template bug
Applying: Fix bug downloading packages
Applying: Add ExternalDocRef to SPDX package
Applying: Regenrate layer SPDX ID to add imageref
Applying: SBOM: Link source SBOM to artifacts SBOM
Applying: Ensure SPDX relationship rednering
Applying: SPDX: Support relationships with remote entities
Applying: License Catalog: Ignore deprecated license IDs
Applying: SPDX: Correctly form layer ref for single img ref
Applying: fix: panic when LICENSE file is not found
Applying: Clean up extra spaces
Applying: lint(revive): Fixup variable declaration warnings
Applying: lint(nolintlint): Enforce stricter settings
Applying: Fix SPDX relationship order btw images and ref
Applying: Support license definition from command flag
Applying: Add possibility to build custom Kubernetes forks
Applying: lint(gosec): Add `nolint`s for SHA1 file integrity usage
Applying: lint(gosec): Guard against decompression bombs when using `io.Copy`
Applying: lint(gosec): Guard against decompression bombs when using filepath.Join
Applying: spdx: Fix panic when unable to dnload err
Applying: SPDX Parser
Applying: SPDX Banner
Applying: lint: Fix typo
Applying: spdx: Document outliner
Applying: bom: document outline subcommand
Applying: Switch to buffered output of the outline
.git/rebase-apply/patch:41: trailing whitespace.
This subcommand draws a tree-like outline to help the user visualize
.git/rebase-apply/patch:42: trailing whitespace.
the structure of the bom. Even when an SBOM represents a graph structure,
.git/rebase-apply/patch:47: trailing whitespace.
attached directly to the root of the document.
warning: 3 lines add whitespace errors.
Applying: spdx: Use -1 to represent unlimited recursion in outline
Applying: spdx: Make regular expressions global to avoid recompiling
Applying: spdx parser: Fix bug in File.Name and NONE Dnld
Applying: spdx: SLSA Provenance export
Applying: Fix several typos/nits in comments

ref: https://dxarts.washington.edu/wiki/git-tricks#copy

// Description updated by @puerco

This commit builds on @jdolitsky 's work to enable JSON output in bom by combining his SPDX-JSON code with one of my branches implementing a new query and filtering feature.

Query Language

The query feature enables adds a query subsystem to bom to enable a new subcommand that allows searching in an sbom. Using the test sbom included in this pr we can now do things like, for example, searching for packages of bash in the sbom:

bom document query pkg/spdx/testdata/nginx.spdx 'name:bash'

 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3
 bash-5.1-2+b3

The initial implementation is rough still but already includes 4 filters (all, purl, depth and name). Documentation of the query language will come soon

JSON Output

Building on Josh's work, we now have a native JSON serializer. The serializer uses the query language filters to flatten the packages and files and render them in the JSON package, the output validates now but I (@puerco) need to add a couple of tests to ensure different SBOM sources render correctly.

Generation now takes the flag added by @jdolitsky and writes JSON and Tag-Value. Example:

❯ bom generate --format=json . > spdx.json
❯ java -jar tmp/tools-java-1.0.4-jar-with-dependencies.jar Verify spdx.json 
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
This SPDX Document is valid.

Check the attached sample SBOM below for a sample

Resolves #103

- **bom now can write SBOMs in JSON!**
- New `bom document query` subcommand allows querying SBOMs using a new filtering language

Signed-off-by: Christopher Angelo Phillips [email protected]

What type of PR is this?

/kind documentation

What this PR does / why we need it:

Add docs clarity for installation.

Which issue(s) this PR fixes:

Special notes for your reviewer:

See Screenshot for updated install command that works

Does this PR introduce a user-facing change?

NONE

What type of PR is this?

/kind feature

What this PR does / why we need it:

• add cloudbuild job and mage rules to build the binaries and images and push those to our bom staging GCP project to be later promoted when needed

1. I need to run a test before we merge this but it is pending some final bits in the GCP project
2. also need to create the postsubmit job which I will do soon

/assign @puerco @justaugustus cc @kubernetes-sigs/release-engineering

Which issue(s) this PR fixes:

None

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

The animation on the main project page is very outdated. Some of the flags are no longer current and the tools supports a lot more features.

We need to record a new animation that can show how to generate an SBOM but also how to check the output, etc.

What happened:

I have a global gitignore that ignores .DS_Store files. When I ran bom against a local git repo, it included .DS_Store in the output.

$ git config --global --get core.excludesFile               
~/.config/git/config

$ git check-ignore --verbose .DS_Store
~/.config/git/ignore:2:.DS_Store      .DS_Store

$ bom generate -d . -n http://example.com/
[...]
Relationship: SPDXRef-Package-my-repo CONTAINS SPDXRef-File-86f7fc8c194db4803d37c5b2feecf09c3ee135fb
FileName: .DS_Store
SPDXID: SPDXRef-File-1f11d692242833beff175ce40424d9f280dd3c8a
FileChecksum: SHA1: 0a9e0aee4f8f8d9bf492bfc64d896d157045fe23
FileChecksum: SHA256: ec746d722904dc246254c17bb76bc7e8579af01ea124f38b06df02200c1d503e
FileChecksum: SHA512: a6f5b29d97fb892fd3ca6189c589f7d1af178d212eebf0294848eb734ec7dbb13d69a09c83419199a0936295f0c7ca936b78898a0b6c9fba1745fd288296aebd
LicenseConcluded: NOASSERTION
LicenseInfoInFile: NONE
FileCopyrightText: NOASSERTION
[...]

I also confirmed this behavior is present for non-standard gitignore files (i.e., not named .gitignore) set locally with (core.excludesFile).

What you expected to happen:

I expected bom to ignore the .DS_Store file, and any others excluded by my global gitignore.

How to reproduce it (as minimally and precisely as possible):

$ git init bom-gitignore-test
Initialized empty Git repository in ~/bom-gitignore-test/.git/

$ cd bom-gitignore-test

$ touch some-file

$ echo some-file > ignores

$ git config --global core.excludesFile ignores # or use --local to only affect this repo

$ git check-ignore --verbose some-file
ignores:1:some-file     some-file

$ bom generate -d . -n http://example.com/
[...]
Relationship: SPDXRef-Package-bom-gitignore-test CONTAINS SPDXRef-File-cbbba3a3cb11930313b724a71f4775324a43ff61
FileName: some-file
SPDXID: SPDXRef-File-45e075645e9d6d3346b37d8e0cd826a0b203f055
FileChecksum: SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
FileChecksum: SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
FileChecksum: SHA512: cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
LicenseConcluded: NOASSERTION
LicenseInfoInFile: NONE
FileCopyrightText: NOASSERTION
[...]

Anything else we need to know?:

Global gitignore files can exist at several paths without being set in the user's gitconfig, so the output of git config --global --get core.excludesFile is not sufficient alone. Here is an excellent gist with some background.

Environment:

• Cloud provider or hardware configuration: Macbook Air M1
• OS (e.g: cat /etc/os-release): MacOS Monterery
• Kernel (e.g. uname -a): darwin 21.2.0
• Others: n/a

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

A new approach to replace the pkg/errors dependency on a package-by-package basis. Concerned packages for this PR: pkg/provenance magefile.go

Which issue(s) this PR fixes:

Refers to: https://github.com/kubernetes-sigs/bom/issues/114

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

None

What type of PR is this?

/kind bug /kind cleanup

What this PR does / why we need it:

This PR improves the licensing code to be more resilient when encountering unexpected license tags in the classifier output. Since v2.0.0, [the classifier will now return a pseudo license tag when finding copyright information] in files. This caused our libraries to return an error when scanning files.

It also fixes another bug where the secondary license list was lost after scanning and classifying a file.

The verbosity of the licensing code has now been reduced to make it more understandable.

Which issue(s) this PR fixes:

Part of https://github.com/kubernetes/release/issues/2729

Special notes for your reviewer:

/assign @jeremyrickard @xmudrii @cpanato

Does this PR introduce a user-facing change?

- Fixed a bug where the secondary license list returned by the classifier was not being returned
- Improved the licensing code to be more resilient to unexpected output from the classifier
- Licensing output is now less verbose. Use `--log-level=debug` to see all messages

Bumps github.com/google/licenseclassifier/v2 from 2.0.0-alpha.1 to 2.0.0-pre5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

What would you like to be added:

SPDX has support for referencing elements defined in external SBOMs and building relationships pointing to them, while we already have support in the libraries for it already, the command line utility does not have a flag to define them.

Why is this needed:

If you want to use Micro SBOMs, there is no way to link them together using only the command line tool.

Signed-off-by: Batuhan Apaydın [email protected] Co-authored-by: Jason Hall [email protected]

What type of PR is this?

/kind feature /kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #135

Special notes for your reviewer:

After a quick chat with @imjasonh, thanks to him, we have agreed that the best way of providing a go binary for an image that is building with ko is changing the image's base with the one of the go binary installed. Fortunately, we have one, https://github.com/distroless/go. distroless.dev/go is an apko-built distroless image that includes go.

cc: @imjasonh

Does this PR introduce a user-facing change?

chore: use different base image to include go

What happened:

An error occurred when creating an SBOM for a container image specified using a digest.

What you expected to happen:

I expected no error to happen when creating an SBOM for a container image specified using a digest.

How to reproduce it (as minimally and precisely as possible):

bom generate --image alpine@sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4

Error:

panic: interface conversion: name.Reference is name.Digest, not name.Tag

goroutine 1 [running]:
sigs.k8s.io/bom/pkg/spdx.refInfoFromIndex(0xc000002140)
	sigs.k8s.io/bom/pkg/spdx/implementation.go:253 +0xed3
sigs.k8s.io/bom/pkg/spdx.getImageReferences({0xc0000de870, 0x4e})
	sigs.k8s.io/bom/pkg/spdx/implementation.go:211 +0x25d
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).PullImagesToArchive(0x0?, {0xc0000de870?, 0x15755c3?}, {0xc0000de960, 0x45})
	sigs.k8s.io/bom/pkg/spdx/implementation.go:359 +0x4c
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).ImageRefToPackage(0xc0000ca000?, {0xc0000de870, 0x4e}, 0x1e?)
	sigs.k8s.io/bom/pkg/spdx/implementation.go:698 +0x172
sigs.k8s.io/bom/pkg/spdx.(*SPDX).ImageRefToPackage(...)
	sigs.k8s.io/bom/pkg/spdx/spdx.go:253
sigs.k8s.io/bom/pkg/spdx.(*defaultDocBuilderImpl).GenerateDoc(0x0?, 0x1988850, 0xc0000c10e0)
	sigs.k8s.io/bom/pkg/spdx/builder.go:234 +0xb6a
sigs.k8s.io/bom/pkg/spdx.(*DocBuilder).Generate(0xc0000edb50?, 0x1?)
	sigs.k8s.io/bom/pkg/spdx/builder.go:96 +0xb5
sigs.k8s.io/bom/cmd/bom/cmd.generateBOM(0xc0000c0fc0)
	sigs.k8s.io/bom/cmd/bom/cmd/generate.go:341 +0x3db
sigs.k8s.io/bom/cmd/bom/cmd.AddGenerate.func1(0xc0001cc000?, {0xc0000c7760?, 0x2?, 0x2?})
	sigs.k8s.io/bom/cmd/bom/cmd/generate.go:146 +0xe8
github.com/spf13/cobra.(*Command).execute(0xc0001cc000, {0xc0000c7740, 0x2, 0x2})
	github.com/spf13/[email protected]/command.go:916 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0x19805e0)
	github.com/spf13/[email protected]/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:968
sigs.k8s.io/bom/cmd/bom/cmd.Execute()
	sigs.k8s.io/bom/cmd/bom/cmd/root.go:71 +0x25
main.main()
	./main.go:24 +0x17

Anything else we need to know?:

Is this the intended behavior? If only tags are supported on purpose, then please close this issue. Thank you for this great tool!

Environment:

OS: M1 Mac w/ macOS 13.0.1 Go: 1.19.2

What happened:

Error when installing using the README installation instructions:

$ go install sigs.k8s.io/bom/cmd/bom
go: 'go install' requires a version when current directory is not in a module
	Try 'go install sigs.k8s.io/bom/cmd/bom@latest' to install the latest version

What you expected to happen:

A successful installation when using go install

How to reproduce it (as minimally and precisely as possible):

$ go install sigs.k8s.io/bom/cmd/bom

Anything else we need to know?:

Adding @latest onto the end of the command solves this problem. A simple PR could likely solve this. I'm glad to submit such a PR.

Environment:

• OS: M1 Mac w/ macOS 13.0.1
• Go: 1.19.2

What happened:

bom does not leverage the local go cache to look for dependency data while generating SBOM in Cilium image build actions.

Generating SBOM describing the source in the Cilium repository using bom takes, on average, 10 minutes. As a result, the CI build time increases by 30 minutes if we generate an SBOM describing the source for all three CI images in Image CI Build and the CI ultimately fails, throwing an error that no space is left on the runner.

In theory, if you run "bom generate" in the same environment where you are building (especially after building), all modules should be there already downloaded, and bom can reuse them. But this does not happen. One thing that bom will not do is download stuff into your go directory. If a module is missing, bom will download it to /tmp/spdx/gomod-scanner/, look at it there, and remove it. Even after performing a "go mod download" before running "bom generate", the downloaded modules are not being picked up by the go interpreter when bom runs.

The downloaded modules are not being picked up by the go interpreter when bom runs: https://github.com/cilium/cilium/actions/runs/3490449396/jobs/5841895937#step:23:1755 for this workflow file.

What you expected to happen:

If bom generate is run in the same environment where you are building (especially after building), all modules should be there already downloaded and bom can reuse them.

Anything else we need to know?:

Discussion on this in K8s slack linked here.

cc @puerco @aanm @nbusseneau

The code is a first draft. The idea is to make it work first, then simplify and make code beautiful.

• [X] Add mage file target
• [x] Modify code to use embed fs.
• [ ] Prettify code
• [ ] Create a job / pipeline to renew the embed FS in the binary distribution

bom currently makes almost 500 requests to download various license data.

INFO Read data for 496 licenses. Downloading.     
INFO Sending GET request to https://spdx.org/licenses/GPL-2.0.json 
INFO Sending GET request to https://spdx.org/licenses/Libpng.json 
INFO Sending GET request to https://spdx.org/licenses/blessing.json 
INFO Sending GET request to https://spdx.org/licenses/VSL-1.0.json 
INFO Sending GET request to https://spdx.org/licenses/BSD-4-Clause-Shortened.json 
INFO Sending GET request to https://spdx.org/licenses/OFL-1.0-no-RFN.json 
.....

Caching these is not an option on docker based installation.

We could fix this by either:

1. Request SPDX to publish these as an archive Or
2. Cloning https://github.com/spdx/license-list-data/tree/master/json/details

(2) is something we could immediately do.

The following command

bom generate -o sbom_ci_pr_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
  bom generate -o sbom_ci_pr_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
  --dirs=. \
  --image=quay.io/cilium/docker-plugin-ci@sha256:b6a2934ca71557db04e88e42548e26cf8f907eadc0eb2fd7c0abc232215b54aa
  bom generate -o sbom_ci_pr_race_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
  --dirs=. \
  --image=quay.io/cilium/docker-plugin-ci@sha256:8348cf7c457e4261469f7313889b51d0976645e4fdb762910639ad569cca555c
  bom generate -o sbom_ci_pr_unstripped_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
  --dirs=. \
  --image=quay.io/cilium/docker-plugin-ci@sha256:2b454f79a725e060602806202126576b45b676e380be502ff4cef24659dab334

results in

panic: interface conversion: name.Reference is name.Digest, not name.Tag

goroutine 1 [running]:
sigs.k8s.io/bom/pkg/spdx.refInfoFromIndex(0xc0020a7540)
	sigs.k8s.io/bom/pkg/spdx/implementation.go:253 +0xed3
sigs.k8s.io/bom/pkg/spdx.getImageReferences({0xc0000308c0, 0x67})
	sigs.k8s.io/bom/pkg/spdx/implementation.go:211 +0x25d
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).PullImagesToArchive(0x0?, {0xc0000308c0?, 0x973a03?}, {0xc00835f6a0, 0x19})
	sigs.k8s.io/bom/pkg/spdx/implementation.go:359 +0x4c
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).ImageRefToPackage(0xc0000a2000?, {0xc0000308c0, 0x67}, 0x1e?)
	sigs.k8s.io/bom/pkg/spdx/implementation.go:698 +0x172
sigs.k8s.io/bom/pkg/spdx.(*SPDX).ImageRefToPackage(...)
	sigs.k8s.io/bom/pkg/spdx/spdx.go:253
sigs.k8s.io/bom/pkg/spdx.(*defaultDocBuilderImpl).GenerateDoc(0x0?, 0xd82b00, 0xc00009cfc0)
	sigs.k8s.io/bom/pkg/spdx/builder.go:234 +0xb6a
sigs.k8s.io/bom/pkg/spdx.(*DocBuilder).Generate(0xc0000b9b50?, 0x1?)
	sigs.k8s.io/bom/pkg/spdx/builder.go:96 +0xb5
sigs.k8s.io/bom/cmd/bom/cmd.generateBOM(0xc00009cea0)
	sigs.k8s.io/bom/cmd/bom/cmd/generate.go:341 +0x3db
sigs.k8s.io/bom/cmd/bom/cmd.AddGenerate.func1(0xc000178000?, {0xc000167100?, 0x4?, 0x4?})
	sigs.k8s.io/bom/cmd/bom/cmd/generate.go:146 +0xe8
github.com/spf13/cobra.(*Command).execute(0xc000178000, {0xc0001670c0, 0x4, 0x4})
	github.com/spf13/[email protected]/command.go:916 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0xd7a3e0)
	github.com/spf13/[email protected]/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:968
sigs.k8s.io/bom/cmd/bom/cmd.Execute()
	sigs.k8s.io/bom/cmd/bom/cmd/root.go:71 +0x25
main.main()
	./main.go:24 +0x17

Version v0.4.1/bom-linux-amd64

as seen in https://github.com/cilium/cilium/actions/runs/3392808891/jobs/5640821590#step:22:202 cc @sandipanpanda @puerco