Thales Maciel
8f4be112c2
Closes the full arc: banger kernel pull + image pull + vm create + vm ssh now works end-to-end against docker.io/library/debian:bookworm with zero manual image building. Generic kernel: - New scripts/make-generic-kernel.sh builds vmlinux from upstream kernel.org sources using Firecracker's official minimal config (configs/firecracker-x86_64-6.1.config). All critical drivers (virtio_blk, virtio_net, ext4, vsock) compiled in — no modules, no initramfs needed. - Published as generic-6.12 in the catalog (kernels.thaloco.com). - catalog.json updated with the new entry. Direct-boot init= override (vm_lifecycle.go): - For images without an initrd (direct-boot / OCI-pulled), banger now passes init=/usr/local/libexec/banger-first-boot on the kernel cmdline. The script runs as PID 1, mounts /proc /sys /dev /run, checks for systemd — if present execs it immediately; if not (container images), installs systemd-sysv + openssh-server via the guest's package manager, then execs systemd. - Also passes kernel-level ip= parameter via BuildBootArgsWithKernelIP so the kernel configures the network interface before init runs (container images don't ship iproute2, so the userspace bootstrap script can't call ip(8)). - Masks dev-ttyS0.device and dev-vdb.device systemd units that otherwise wait 90s for udev events that never fire in Firecracker guests started from container rootfses. first-boot.sh rewritten as universal init wrapper: - Works as PID 1 (mounts essential filesystems) OR as a systemd oneshot (existing behavior). - Installs both systemd-sysv AND openssh-server (container images have neither). - Dispatch updated: debian, alpine, fedora, arch, opensuse families + ID_LIKE fallback. All tests updated. Opencode capability skip for direct-boot images: - The opencode readiness check (WaitReady on vsock port 4096) now returns nil for images without an initrd, since pulled container images don't ship the opencode service. Without this, the VM would be marked as error for lacking an opinionated add-on. Docs: README and kernel-catalog.md updated to recommend generic-6.12 as the default kernel for OCI-pulled images. AGENTS.md notes the new build script. Verified live: - banger kernel pull generic-6.12 - banger image pull docker.io/library/debian:bookworm --kernel-ref generic-6.12 - banger vm create --image debian-bookworm --name testbox --nat - banger vm ssh testbox -- "id; uname -r; systemctl is-active banger-vsock-agent" → uid=0(root), kernel 6.12.8, Debian bookworm, vsock-agent active, sshd running, SSH working. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
154 lines
6.3 KiB
Markdown
154 lines
6.3 KiB
Markdown
# Kernel catalog
|
|
|
|
The kernel catalog ships pre-built Firecracker-ready kernel bundles so users
|
|
don't have to compile anything. The catalog is embedded into the banger
|
|
binary and updated each release.
|
|
|
|
End-user flow:
|
|
|
|
```bash
|
|
banger kernel list --available # browse the catalog
|
|
banger kernel pull void-6.12 # download a bundle (no sudo, no make)
|
|
banger image register --name void --rootfs … --kernel-ref void-6.12
|
|
```
|
|
|
|
## Architecture
|
|
|
|
Two parts:
|
|
|
|
1. **`internal/kernelcat/catalog.json`** — a JSON manifest embedded into the
|
|
banger binary via `go:embed`. Each entry carries a name, distro, arch,
|
|
kernel version, tarball URL, and tarball SHA256. Updating the catalog
|
|
means editing this file in the repo and rebuilding banger.
|
|
|
|
2. **Tarballs at `https://kernels.thaloco.com/`** — Cloudflare R2 bucket
|
|
`banger-kernels`, fronted by a public custom domain. Each tarball is
|
|
`<name>-<arch>.tar.zst` and contains `vmlinux`, optional `initrd.img`,
|
|
and an optional `modules/` tree at the archive root.
|
|
|
|
The `banger kernel pull` flow streams the tarball, verifies its SHA256
|
|
against the embedded catalog entry, decompresses it (zstd), extracts it
|
|
into `~/.local/state/banger/kernels/<name>/`, and writes a manifest. Path
|
|
traversal entries and unsafe symlinks are rejected.
|
|
|
|
## Kernel types
|
|
|
|
**`generic-<version>`** — built from upstream kernel.org sources with
|
|
Firecracker's official config. All essential drivers (virtio_blk,
|
|
virtio_net, ext4, vsock) compiled in — no modules, no initramfs. This
|
|
is the recommended kernel for OCI-pulled images (Debian, Ubuntu,
|
|
Fedora, etc.). Build with `scripts/make-generic-kernel.sh`.
|
|
|
|
**`void-<version>` / `alpine-<version>`** — distro-specific kernels
|
|
built from Void/Alpine package repos. Include initramfs + modules.
|
|
These are for the `make rootfs-void` / `make rootfs-alpine` manual
|
|
flows where the initramfs is paired with its matching rootfs.
|
|
|
|
## Adding or updating an entry
|
|
|
|
The repo has no CI for kernel publishing yet. Catalog updates are manual
|
|
and infrequent (kernel version bumps every few weeks at most).
|
|
|
|
```bash
|
|
# 1. Build the kernel locally with the existing helper.
|
|
scripts/make-generic-kernel.sh # or: make void-kernel / make alpine-kernel
|
|
|
|
# 2. Import it into the local catalog so the canonical layout exists.
|
|
banger kernel import generic-6.12 \
|
|
--from build/manual/generic-kernel \
|
|
--distro generic \
|
|
--arch x86_64
|
|
|
|
# 3. Package, upload, patch catalog.json.
|
|
scripts/publish-kernel.sh generic-6.12 \
|
|
--description "Generic Firecracker kernel 6.12 (all drivers built-in, no initrd)"
|
|
|
|
# 4. Review and commit the catalog change.
|
|
git diff -- internal/kernelcat/catalog.json
|
|
git add internal/kernelcat/catalog.json
|
|
git commit -m 'kernel catalog: add/update generic-6.12'
|
|
|
|
# 5. Rebuild so the new catalog is embedded.
|
|
make build
|
|
```
|
|
|
|
`scripts/publish-kernel.sh` reads the locally-imported entry under
|
|
`~/.local/state/banger/kernels/<name>/`, builds a tar+zstd archive, uploads
|
|
it to R2 via `rclone`, HEAD-checks the public URL, and patches
|
|
`internal/kernelcat/catalog.json` with the new URL, SHA256, and size.
|
|
|
|
Environment overrides if the defaults need to change:
|
|
`RCLONE_REMOTE`, `RCLONE_BUCKET`, `BASE_URL`, `BANGER_KERNELS_DIR`.
|
|
|
|
## Removing an entry
|
|
|
|
1. Delete the line from `internal/kernelcat/catalog.json` and commit.
|
|
2. Delete the tarball from R2: `rclone delete r2:banger-kernels/<name>-<arch>.tar.zst`.
|
|
3. Rebuild banger.
|
|
|
|
Already-pulled local copies on user machines are not invalidated — they
|
|
keep working until the user runs `banger kernel rm <name>`. That's
|
|
intentional: pulling is idempotent, removing should not break anyone in
|
|
the middle of a workflow.
|
|
|
|
## Versioning conventions
|
|
|
|
- **Entry names**: `<distro>-<major.minor>` (e.g. `void-6.12`,
|
|
`alpine-3.23`). The major.minor is the kernel line, not the distro
|
|
release. Patch-level bumps reuse the entry name and replace the
|
|
tarball; minor bumps create a new entry (`void-6.13`).
|
|
- **Architecture**: only `x86_64` is published today. The `arch` field in
|
|
the catalog schema is additive — adding `arm64` later is a config
|
|
change, not a schema change.
|
|
- **Tarball layout**: contents at the archive root (no top-level
|
|
versioned directory). `vmlinux` is required; `initrd.img` and
|
|
`modules/` are optional. Symlinks inside `modules/` are allowed but
|
|
must resolve within the archive.
|
|
|
|
## Trust model
|
|
|
|
The embedded `catalog.json` carries the SHA256 of each tarball. `banger
|
|
kernel pull` rejects any download whose hash doesn't match. This protects
|
|
against transport corruption and against an attacker swapping a tarball
|
|
on R2 without also pushing a banger release.
|
|
|
|
It does **not** protect against a compromise of the banger source repo
|
|
itself — an attacker who can land a commit can change both the catalog
|
|
SHA256 and the tarball. GPG/sigstore signing is deferred until banger is
|
|
public and the threat model justifies the operational overhead.
|
|
|
|
## Hosting
|
|
|
|
Tarballs live in Cloudflare R2 (bucket `banger-kernels`), served at the
|
|
custom domain `kernels.thaloco.com`. The bucket is publicly readable;
|
|
writes require the `banger-kernels-publish` API token (kept locally,
|
|
never committed). R2's free tier covers the expected traffic comfortably
|
|
(zero egress fees, generous storage).
|
|
|
|
If hosting ever moves, catalog entries can be migrated by reuploading the
|
|
tarballs and editing the URLs in `catalog.json` — no other code changes
|
|
required.
|
|
|
|
## Tech debt: kernel-build scripts
|
|
|
|
`scripts/make-void-kernel.sh` and `scripts/make-alpine-kernel.sh` are
|
|
procedural bash that fetches and patches per-distro kernel sources.
|
|
Each new distro means a new bespoke script. They're "good enough"
|
|
because catalog refreshes are infrequent and only the maintainer runs
|
|
them, but they are the bottleneck if the catalog ever wants to grow
|
|
beyond two distros.
|
|
|
|
A future iteration should:
|
|
|
|
- Move kernel acquisition into a Go (or at least uniform) tool with a
|
|
per-distro plugin/config rather than per-distro scripts.
|
|
- Encode kernel config and required modules declaratively so a Debian
|
|
or Fedora target is a config addition, not a new script.
|
|
- Run unattended in CI once banger goes public — the manual
|
|
`scripts/publish-kernel.sh` flow scales until then.
|
|
|
|
Until that happens, `make lint-shell` only runs at `--severity=error`.
|
|
Tightening to `--severity=warning` would surface real issues in the
|
|
legacy build scripts (mostly `sudo cat > file` redirects and
|
|
heredoc-quoting concerns); fixing those is a prerequisite to bumping
|
|
the lint floor.
|