Phase 5: kernel catalog publish flow + docs
Manual publish flow for the kernel catalog, designed for the current no-CI, private-repo state of banger. scripts/publish-kernel.sh <name>: - Reads $BANGER_KERNELS_DIR/<name>/ (the canonical layout produced by `banger kernel import`). - Pulls distro / arch / kernel_version from the local manifest. - Packages vmlinux + optional initrd.img + optional modules/ as <name>-<arch>.tar.zst with zstd -19. - Computes sha256 + size. - rclone copyto -> r2:banger-kernels/<file>. - HEAD-checks https://kernels.thaloco.com/<file> to catch public-access misconfig before declaring success. - jq-patches internal/kernelcat/catalog.json: replaces any prior entry with the same name, then sorts entries by name. - Prints next-step git+make commands; does not commit or rebuild automatically. Environment overrides RCLONE_REMOTE / RCLONE_BUCKET / BASE_URL / BANGER_KERNELS_DIR for non-default setups. docs/kernel-catalog.md covers the architecture (embedded JSON + external tarballs), end-user flow, the add/update/remove playbook, naming and tarball-layout conventions, the trust model (sha256 in embedded catalog catches transport/swap; no signing yet), and where the bucket lives. README.md gains a kernel-catalog example next to the existing image register example. AGENTS.md points at publish-kernel.sh and the docs. .gitignore now excludes .env so accidental drops of R2 credentials don't follow into commits. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
f0668ee598
commit
fa95849f5a
5 changed files with 272 additions and 0 deletions
117
docs/kernel-catalog.md
Normal file
117
docs/kernel-catalog.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
# Kernel catalog
|
||||
|
||||
The kernel catalog ships pre-built Firecracker-ready kernel bundles so users
|
||||
don't have to compile anything. The catalog is embedded into the banger
|
||||
binary and updated each release.
|
||||
|
||||
End-user flow:
|
||||
|
||||
```bash
|
||||
banger kernel list --available # browse the catalog
|
||||
banger kernel pull void-6.12 # download a bundle (no sudo, no make)
|
||||
banger image register --name void --rootfs … --kernel-ref void-6.12
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
Two parts:
|
||||
|
||||
1. **`internal/kernelcat/catalog.json`** — a JSON manifest embedded into the
|
||||
banger binary via `go:embed`. Each entry carries a name, distro, arch,
|
||||
kernel version, tarball URL, and tarball SHA256. Updating the catalog
|
||||
means editing this file in the repo and rebuilding banger.
|
||||
|
||||
2. **Tarballs at `https://kernels.thaloco.com/`** — Cloudflare R2 bucket
|
||||
`banger-kernels`, fronted by a public custom domain. Each tarball is
|
||||
`<name>-<arch>.tar.zst` and contains `vmlinux`, optional `initrd.img`,
|
||||
and an optional `modules/` tree at the archive root.
|
||||
|
||||
The `banger kernel pull` flow streams the tarball, verifies its SHA256
|
||||
against the embedded catalog entry, decompresses it (zstd), extracts it
|
||||
into `~/.local/state/banger/kernels/<name>/`, and writes a manifest. Path
|
||||
traversal entries and unsafe symlinks are rejected.
|
||||
|
||||
## Adding or updating an entry
|
||||
|
||||
The repo has no CI for kernel publishing yet. Catalog updates are manual
|
||||
and infrequent (kernel version bumps every few weeks at most).
|
||||
|
||||
```bash
|
||||
# 1. Build the kernel locally with the existing helper.
|
||||
make void-kernel # or: make alpine-kernel
|
||||
|
||||
# 2. Import it into the local catalog so the canonical layout exists.
|
||||
banger kernel import void-6.12 \
|
||||
--from build/manual/void-kernel \
|
||||
--distro void \
|
||||
--arch x86_64
|
||||
|
||||
# 3. Package, upload, patch catalog.json.
|
||||
scripts/publish-kernel.sh void-6.12 \
|
||||
--description "Void Linux 6.12 kernel for Firecracker microVMs"
|
||||
|
||||
# 4. Review and commit the catalog change.
|
||||
git diff -- internal/kernelcat/catalog.json
|
||||
git add internal/kernelcat/catalog.json
|
||||
git commit -m 'kernel catalog: add/update void-6.12'
|
||||
|
||||
# 5. Rebuild so the new catalog is embedded.
|
||||
make build
|
||||
```
|
||||
|
||||
`scripts/publish-kernel.sh` reads the locally-imported entry under
|
||||
`~/.local/state/banger/kernels/<name>/`, builds a tar+zstd archive, uploads
|
||||
it to R2 via `rclone`, HEAD-checks the public URL, and patches
|
||||
`internal/kernelcat/catalog.json` with the new URL, SHA256, and size.
|
||||
|
||||
Environment overrides if the defaults need to change:
|
||||
`RCLONE_REMOTE`, `RCLONE_BUCKET`, `BASE_URL`, `BANGER_KERNELS_DIR`.
|
||||
|
||||
## Removing an entry
|
||||
|
||||
1. Delete the line from `internal/kernelcat/catalog.json` and commit.
|
||||
2. Delete the tarball from R2: `rclone delete r2:banger-kernels/<name>-<arch>.tar.zst`.
|
||||
3. Rebuild banger.
|
||||
|
||||
Already-pulled local copies on user machines are not invalidated — they
|
||||
keep working until the user runs `banger kernel rm <name>`. That's
|
||||
intentional: pulling is idempotent, removing should not break anyone in
|
||||
the middle of a workflow.
|
||||
|
||||
## Versioning conventions
|
||||
|
||||
- **Entry names**: `<distro>-<major.minor>` (e.g. `void-6.12`,
|
||||
`alpine-3.23`). The major.minor is the kernel line, not the distro
|
||||
release. Patch-level bumps reuse the entry name and replace the
|
||||
tarball; minor bumps create a new entry (`void-6.13`).
|
||||
- **Architecture**: only `x86_64` is published today. The `arch` field in
|
||||
the catalog schema is additive — adding `arm64` later is a config
|
||||
change, not a schema change.
|
||||
- **Tarball layout**: contents at the archive root (no top-level
|
||||
versioned directory). `vmlinux` is required; `initrd.img` and
|
||||
`modules/` are optional. Symlinks inside `modules/` are allowed but
|
||||
must resolve within the archive.
|
||||
|
||||
## Trust model
|
||||
|
||||
The embedded `catalog.json` carries the SHA256 of each tarball. `banger
|
||||
kernel pull` rejects any download whose hash doesn't match. This protects
|
||||
against transport corruption and against an attacker swapping a tarball
|
||||
on R2 without also pushing a banger release.
|
||||
|
||||
It does **not** protect against a compromise of the banger source repo
|
||||
itself — an attacker who can land a commit can change both the catalog
|
||||
SHA256 and the tarball. GPG/sigstore signing is deferred until banger is
|
||||
public and the threat model justifies the operational overhead.
|
||||
|
||||
## Hosting
|
||||
|
||||
Tarballs live in Cloudflare R2 (bucket `banger-kernels`), served at the
|
||||
custom domain `kernels.thaloco.com`. The bucket is publicly readable;
|
||||
writes require the `banger-kernels-publish` API token (kept locally,
|
||||
never committed). R2's free tier covers the expected traffic comfortably
|
||||
(zero egress fees, generous storage).
|
||||
|
||||
If hosting ever moves, catalog entries can be migrated by reuploading the
|
||||
tarballs and editing the URLs in `catalog.json` — no other code changes
|
||||
required.
|
||||
Loading…
Add table
Add a link
Reference in a new issue