thaloco/banger
1
Fork 0
Code Pull requests Releases Packages Activity Actions

docs: promote vm run + image catalog as the happy path

Browse source 
Lead the README with `banger vm run` (one command, auto-pull default
image + kernel from the catalogs), move `image register` / `image
build` / OCI-pull to a "power-user flows" section. Golden-image
content from customize.sh moves to the golden-image Dockerfile story.

New `docs/image-catalog.md` mirrors `docs/kernel-catalog.md` — the
bundle format, content-addressed filenames, publish flow, trust
model, R2 hosting. Cross-links with oci-import.md.

`docs/oci-import.md` refactored to document the OCI-pull path as the
fallthrough for arbitrary registry refs (it's the secondary path now
that the catalog covers the headline debian-bookworm case). Phase A
caveats removed — ownership fixup, agent injection, and first-boot
sshd install all landed.

AGENTS.md: promotes `vm run` as the smoke-test primitive, notes the
default-image auto-pull behaviour, and points at both catalog docs.

README shrinks 330 → 198 lines, mostly by removing the experimental
void/alpine sections (those flows still work as advanced scripts but
the README no longer advertises them).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Thales Maciel 2026-04-18 15:33:30 -03:00
parent 75baf2e415
commit 8029b2e1bc
No known key found for this signature in database
GPG key ID: 33112E6833C34679
4 changed files with 404 additions and 450 deletions
Download patch file Download diff file Expand all files Collapse all files

21
AGENTS.md
View file

@ -7,7 +7,9 @@ Always run `make build` before commit.
- `cmd/banger` and `cmd/bangerd` are the main user entrypoints. - `cmd/banger` and `cmd/bangerd` are the main user entrypoints.
- `internal/` contains the daemon, CLI, RPC, storage, Firecracker integration, guest helpers, and the experimental web UI. - `internal/` contains the daemon, CLI, RPC, storage, Firecracker integration, guest helpers, and the experimental web UI.
- `internal/daemon/` is the composition root; pure helpers live in its subpackages (`opstate`, `dmsnap`, `fcproc`, `imagemgr`, `session`, `workspace`). See `internal/daemon/ARCHITECTURE.md`. - `internal/daemon/` is the composition root; pure helpers live in its subpackages (`opstate`, `dmsnap`, `fcproc`, `imagemgr`, `session`, `workspace`). See `internal/daemon/ARCHITECTURE.md`.
- `scripts/` contains explicit manual helper workflows for rootfs and kernel preparation. - `internal/imagecat/` and `internal/kernelcat/` embed the image + kernel catalogs.
- `images/golden/` is the Dockerfile for the `debian-bookworm` catalog entry.
- `scripts/` contains manual helper workflows for rootfs, kernel, and bundle preparation.
- `build/bin/` is the canonical source-checkout build output. - `build/bin/` is the canonical source-checkout build output.
- `build/manual/` is the canonical source-checkout location for manual rootfs/kernel artifacts. - `build/manual/` is the canonical source-checkout location for manual rootfs/kernel artifacts.
@ -17,19 +19,20 @@ Always run `make build` before commit.
- `make test` runs `go test ./...`. - `make test` runs `go test ./...`.
- `make lint` runs `gofmt -l`, `go vet ./...`, and `shellcheck --severity=error` on `scripts/*.sh`. Run before commits. - `make lint` runs `gofmt -l`, `go vet ./...`, and `shellcheck --severity=error` on `scripts/*.sh`. Run before commits.
- `./build/bin/banger doctor` checks host readiness. - `./build/bin/banger doctor` checks host readiness.
- `./build/bin/banger image build --from-image <image>` builds a managed image from an existing registered image. - `./build/bin/banger vm run` is the primary user-facing entry point — auto-pulls the default image + kernel from the catalogs if missing.
- `./build/bin/banger image pull <name>` uses the bundle catalog (fast) when `<name>` is a catalog entry, or falls through to the OCI path for arbitrary registry refs. See `docs/image-catalog.md` and `docs/oci-import.md`.
- `./build/bin/banger image register ...` registers an unmanaged host-side image stack. - `./build/bin/banger image register ...` registers an unmanaged host-side image stack.
- `./build/bin/banger image build --from-image <image>` builds a managed image from an existing one.
- `./build/bin/banger image promote <image>` copies an unmanaged image into daemon-owned managed artifacts. - `./build/bin/banger image promote <image>` copies an unmanaged image into daemon-owned managed artifacts.
- `scripts/make-generic-kernel.sh` builds a Firecracker-optimized vmlinux from upstream sources (no initrd, all drivers built-in). This is the recommended kernel for OCI-pulled images. - `scripts/make-generic-kernel.sh` builds a Firecracker-optimized vmlinux from upstream sources. `scripts/publish-kernel.sh <name>` publishes it to the kernel catalog.
- `make void-kernel`, `make rootfs-void`, and `make void-register` drive the experimental Void flow under `./build/manual`. - `scripts/publish-golden-image.sh` rebuilds + publishes the golden image bundle and patches the image catalog.
- `scripts/publish-kernel.sh <name>` packages a locally-imported kernel and uploads it to the catalog; see `docs/kernel-catalog.md`.
- `banger image pull <oci-ref> --kernel-ref <name>` pulls a rootfs from any OCI registry; see `docs/oci-import.md` (experimental — file-ownership caveat).
## Image Model ## Image Model
- Managed images own the full boot set: rootfs, optional work-seed, kernel, optional initrd, and optional modules. - Managed images own the full boot set: rootfs, optional work-seed, kernel, optional initrd, and optional modules.
- There is no runtime bundle and no auto-registered default image from disk paths. - The image catalog ships pre-built bundles. `vm run` auto-pulls the default catalog entry; `image pull <name>` can be invoked explicitly.
- `default_image_name` selects a registered image only. - `default_image_name` defaults to `debian-bookworm`. On miss, the daemon auto-pulls from `imagecat` before surfacing "not found".
- Kernel references follow the same auto-pull pattern against `kernelcat`.
## Config ## Config
@ -50,7 +53,7 @@ Always run `make build` before commit.
## Testing Guidance ## Testing Guidance
- Primary automated coverage is `go test ./...`. - Primary automated coverage is `go test ./...`.
- For lifecycle changes, smoke-test with `vm create`, `vm ssh`, `vm stop`, and `vm delete`. - For lifecycle changes, smoke-test with `vm run` end-to-end (covers create + start + boot + ssh).
- If guest provisioning changes, document whether existing images must be rebuilt or recreated. - If guest provisioning changes, document whether existing images must be rebuilt or recreated.
## Security ## Security

448
README.md
View file

@ -1,330 +1,198 @@
# banger # banger
`banger` manages Firecracker development VMs with a local daemon, managed image artifacts, and an experimental localhost web UI. One-command development sandboxes on Firecracker microVMs.
## Quick start
```bash
make install
banger vm run --name sandbox
```
`banger vm run` auto-pulls the default golden image (Debian bookworm
with systemd, sshd, Docker CE, git, jq, mise, and the usual dev tools)
and kernel from the embedded catalog if they aren't already local,
creates a VM, starts it, and drops you into an interactive ssh
session. First run takes a couple minutes (bundle download);
subsequent `vm run`s are seconds.
## Requirements ## Requirements
- Linux with `/dev/kvm` - Linux with `/dev/kvm`
- `sudo` - `sudo`
- Firecracker installed on `PATH`, or `firecracker_bin` set in config - Firecracker on `PATH`, or `firecracker_bin` set in config
- The usual host tools checked by `./build/bin/banger doctor` - host tools checked by `banger doctor`
`banger` now owns complete managed image sets. A managed image includes: ## Build + install
- `rootfs`
- optional `work-seed`
- `kernel`
- optional `initrd`
- optional `modules`
There is no runtime bundle anymore.
## Build
```bash
make build
```
This writes:
- `./build/bin/banger`
- `./build/bin/bangerd`
- `./build/bin/banger-vsock-agent`
## Install
```bash ```bash
make install make install
``` ```
That installs: Installs:
- `banger` - `banger` (CLI)
- `bangerd` - `bangerd` (daemon, auto-starts on first CLI call)
- the `banger-vsock-agent` companion helper under `../lib/banger/` - `banger-vsock-agent` (companion, under `$PREFIX/lib/banger/`)
## `vm run`
One command, three modes:
```bash
banger vm run # bare sandbox — drops into ssh
banger vm run ./repo # workspace at /root/repo — drops into ssh
banger vm run ./repo -- make test # workspace + run command, exit with its status
```
- Bare mode gives you a clean shell.
- Workspace mode (with a path) copies the repo's tracked + untracked
non-ignored files into `/root/repo` and kicks off a best-effort
mise tooling bootstrap from the repo's `.mise.toml` /
`.tool-versions`. Log: `/root/.cache/banger/vm-run-tooling-<repo>.log`.
- Command mode (`-- <cmd>`) runs the command in the guest; exit code
propagates through `banger`.
Disconnecting from an interactive session leaves the VM running. Use
`vm stop` / `vm delete` to clean up.
`--branch` and `--from` apply only to workspace mode.
## Image catalog
`banger image pull <name>` resolves `<name>` in the embedded catalog
and fetches a pre-built bundle (rootfs.ext4 + manifest, tar+zstd). The
kernel referenced by the manifest auto-pulls too. `vm run` calls this
for you on demand.
Today's catalog:
| Name | Distro | Kernel |
|------|--------|--------|
| `debian-bookworm` | Debian 12 slim + sshd + docker + dev tools | `generic-6.12` |
The catalog ships embedded in the banger binary. See
[`docs/image-catalog.md`](docs/image-catalog.md) for maintenance.
## Power-user flows
Skip this section if `vm run` is enough.
### `vm create` — low-level primitive
For scripting or `--no-start` provisioning:
```bash
banger vm create --image debian-bookworm --name testbox --no-start
banger vm start testbox
banger vm ssh testbox
banger vm stop testbox
```
### `image pull <oci-ref>` — arbitrary container images
For images outside the catalog, pull from any OCI registry:
```bash
banger image pull docker.io/library/alpine:3.20 --kernel-ref generic-6.12
```
Layers are flattened, ownership is fixed, banger's guest agents are
injected, and a first-boot service installs `openssh-server` via the
guest's package manager. See [`docs/oci-import.md`](docs/oci-import.md)
for supported distros and caveats.
### `image register` — existing host-side stack
```bash
banger image register --name base \
--rootfs /abs/path/rootfs.ext4 \
--kernel-ref generic-6.12
```
### `image build --from-image` — derived images
```bash
banger image build --name devbox --from-image debian-bookworm --docker
```
Spins up a transient VM from a base image, applies opinionated
customisation (mise, claude, pi, tmux plugins), saves a new managed
image.
### Workspace + session primitives
Long-lived guest commands managed by the daemon, attachable over a
local Unix socket bridge:
```bash
banger vm workspace prepare <vm> ./other-repo --guest-path /root/repo
banger vm session start <vm> --name planner --cwd /root/repo --stdin-mode pipe -- pi --mode rpc
banger vm session attach <vm> planner
banger vm session logs <vm> planner --stream stderr
banger vm session stop <vm> planner
```
For ACP-aware host tooling: `banger vm acp <vm>` bridges stdio to
guest `opencode acp` over SSH.
## Config ## Config
Config lives at `~/.config/banger/config.toml`. Config lives at `~/.config/banger/config.toml`. All keys optional.
Supported keys: Commonly set:
- `log_level` - `default_image_name` — image to use when `--image` is omitted
- `web_listen_addr` (defaults to `debian-bookworm`, auto-pulled from the catalog if not
- `firecracker_bin` local).
- `ssh_key_path` - `ssh_key_path` — host SSH key; if unset banger creates
- `default_image_name` `~/.config/banger/ssh/id_ed25519`.
- `auto_stop_stale_after` - `firecracker_bin` — override the auto-resolved `PATH` lookup.
- `stats_poll_interval` - `web_listen_addr` — experimental web UI (default
- `metrics_poll_interval` `127.0.0.1:7777`; set to `""` to disable).
- `bridge_name` - Network: `bridge_name`, `bridge_ip`, `cidr`, `tap_pool_size`,
- `bridge_ip` `default_dns`.
- `cidr`
- `tap_pool_size`
- `default_dns`
If `ssh_key_path` is unset, banger creates and uses: Full key list in `internal/config/config.go`.
- `~/.config/banger/ssh/id_ed25519` ## Credential sync
`default_image_name` now only means “use this registered image when `vm create` omits `--image`”. The daemon does not auto-register images from host paths. If these host auth files exist, banger syncs them into the guest at
VM start:
## Core Workflow | Host | Guest |
|------|-------|
| `~/.local/share/opencode/auth.json` | `/root/.local/share/opencode/auth.json` |
| `~/.claude/.credentials.json` | `/root/.claude/.credentials.json` |
| `~/.pi/agent/auth.json` | `/root/.pi/agent/auth.json` |
Check the host: Host-side changes take effect after the VM restarts. Session/history
directories are not copied.
```bash
./build/bin/banger doctor
```
Register an existing host-side image stack:
```bash
./build/bin/banger image register \
--name base \
--rootfs /abs/path/rootfs.ext4 \
--kernel /abs/path/vmlinux \
--initrd /abs/path/initrd.img \
--modules /abs/path/modules
```
Or pull a pre-built kernel from the catalog and reference it by name:
```bash
./build/bin/banger kernel list --available
./build/bin/banger kernel pull generic-6.12
./build/bin/banger image register \
--name base \
--rootfs /abs/path/rootfs.ext4 \
--kernel-ref generic-6.12
```
See [`docs/kernel-catalog.md`](docs/kernel-catalog.md) for catalog
maintenance.
Or pull a rootfs directly from any OCI registry (Docker Hub, GHCR, …):
```bash
./build/bin/banger image pull docker.io/library/debian:bookworm \
--kernel-ref generic-6.12
```
`image pull` downloads the image, flattens its layers into an ext4
rootfs, applies tar-header ownership via debugfs, and pre-injects
banger's guest agents (vsock agent + network bootstrap + a first-boot
unit that installs `openssh-server` via the guest's native package
manager). Boots as a banger VM directly, no `image build` step
required. See [`docs/oci-import.md`](docs/oci-import.md) for
supported distros and current limitations.
Build a managed image from an existing registered image:
```bash
./build/bin/banger image build \
--name devbox \
--from-image base \
--docker
```
Promote an unmanaged image into daemon-owned managed artifacts:
```bash
./build/bin/banger image promote base
```
Spin up a sandbox VM and drop straight into it:
```bash
./build/bin/banger vm run # bare sandbox, interactive ssh
./build/bin/banger vm run ../some-repo # workspace at /root/repo, interactive ssh
./build/bin/banger vm run ../some-repo -- make test # workspace, run command, exit with its status
```
`vm run` creates a VM, prepares a workspace if you pass a path, and then either drops you into an interactive ssh session or runs the `--`-delimited command to completion. The command's exit code propagates through `banger`. Disconnecting from the interactive session leaves the VM running; use `vm stop` / `vm delete` to clean up.
When you pass a path, `vm run` copies a git checkout plus tracked and untracked non-ignored files into `/root/repo`, then kicks off a best-effort `mise` tooling bootstrap that runs asynchronously inside the guest (log at `/root/.cache/banger/vm-run-tooling-<repo>.log`). The bootstrap is skipped in bare and command modes. Flags like `--branch` and `--from` require a path.
For scripting or lower-level control, `vm create` remains available as a primitive (use `--no-start` when you just want to provision):
```bash
./build/bin/banger vm create --image devbox --name testbox --no-start
./build/bin/banger vm start testbox
./build/bin/banger vm ssh testbox
./build/bin/banger vm stop testbox
```
`vm create` stays synchronous by default, but on a TTY it now shows live progress until the VM is fully ready.
For ACP-aware host tools, `./build/bin/banger vm acp <vm-name>` bridges stdio to guest `opencode acp` over SSH. It uses `/root/repo` when that checkout exists, otherwise `/root`, and `--cwd` lets you override the guest working directory explicitly.
If you want reusable orchestration primitives instead of the `vm run` convenience flow, use the daemon-backed workspace and session commands directly:
```bash
./build/bin/banger vm workspace prepare <vm-name>
./build/bin/banger vm workspace prepare <vm-name> ../other-repo --guest-path /root/repo --readonly
./build/bin/banger vm session start <vm-name> --name planner --cwd /root/repo --stdin-mode pipe -- pi --mode rpc --no-session
./build/bin/banger vm session list <vm-name>
./build/bin/banger vm session attach <vm-name> planner
./build/bin/banger vm session logs <vm-name> planner --stream stderr
./build/bin/banger vm session stop <vm-name> planner
```
`vm workspace prepare` materializes a local git checkout into a running VM. The default guest path is `/root/repo` and the default mode is a shallow metadata copy plus tracked and untracked non-ignored overlay. Repositories with git submodules must use `--mode full_copy`; the metadata-based modes still reject them.
`vm session start` creates a daemon-managed long-lived guest command. The daemon preflights that the requested guest `cwd` exists and that the main command, plus any repeated `--require-command` entries, exist in guest `PATH` before launch. Use `--stdin-mode pipe` when you need live `attach`; otherwise use the default detached mode and inspect sessions with `list`, `show`, `logs`, `stop`, and `kill`.
`vm session attach` is currently exclusive and same-host only. The daemon exposes a local Unix socket bridge using `stdio_mux_v1`, so only one active attach is allowed at a time. Pipe-mode sessions keep enough guest-side state for the daemon to rebuild that bridge after a daemon restart.
## Web UI (experimental) ## Web UI (experimental)
`bangerd` serves an experimental local web UI by default at: `bangerd` serves a local web UI at `http://127.0.0.1:7777` by default.
Convenient for local observability, **not a stable interface**. Do
- `http://127.0.0.1:7777` not expose the listen address to a shared network.
The UI is convenient for local observability but is **not a stable or
supported interface**. Its endpoints, layout, and behaviour may change
without notice, and it has not been hardened for anything beyond single-user
localhost use. Do not expose the listen address to a shared network.
See the effective URL with:
```bash
./build/bin/banger daemon status
```
Disable it with:
```toml
web_listen_addr = ""
```
## Guest Services
Provisioned glibc-backed images include:
- `banger-vsock-agent`
- guest networking bootstrap
- `mise`
- `opencode`
- `claude`
- `pi`
- a default guest `opencode` service on `0.0.0.0:4096`
Alpine currently remains `opencode`-only.
If these host auth files exist, `banger` syncs them into the guest on VM start:
- `~/.local/share/opencode/auth.json` -> `/root/.local/share/opencode/auth.json`
- `~/.claude/.credentials.json` -> `/root/.claude/.credentials.json`
- `~/.pi/agent/auth.json` -> `/root/.pi/agent/auth.json`
Changes on the host take effect after the VM is restarted. Session/history directories are not copied.
From the host:
```bash
./build/bin/banger vm ports testbox
opencode attach http://<guest-ip>:4096
```
## Manual Helpers
The shell helpers are now explicit manual workflows under `./build/manual`.
Rebuild a Debian-style manual rootfs:
```bash
make rootfs ARGS='--base-rootfs /abs/path/rootfs.ext4 --kernel /abs/path/vmlinux --initrd /abs/path/initrd.img --modules /abs/path/modules'
```
The output lands in:
- `./build/manual/rootfs-docker.ext4`
- `./build/manual/rootfs-docker.work-seed.ext4`
## Experimental Void Flow
Stage a Void kernel:
```bash
make void-kernel
```
Build the experimental Void rootfs:
```bash
make rootfs-void
```
Register it:
```bash
make void-register
```
That flow uses:
- `./build/manual/void-kernel/`
- `./build/manual/rootfs-void.ext4`
- `./build/manual/rootfs-void.work-seed.ext4`
## Experimental Alpine Flow
Stage an Alpine virt kernel:
```bash
make alpine-kernel
```
Build the experimental Alpine rootfs:
```bash
make rootfs-alpine
```
Register it:
```bash
make alpine-register
```
Create a VM from it:
```bash
./build/bin/banger vm create --image alpine --name alpine-dev
```
That flow uses:
- `./build/manual/alpine-kernel/`
- `./build/manual/rootfs-alpine.ext4`
- `./build/manual/rootfs-alpine.work-seed.ext4`
The experimental Alpine flow stages a pinned Alpine release by default. Override
that pin with `ALPINE_RELEASE=...` when running the `make alpine-kernel` and
`make rootfs-alpine` helpers if you need a different patch release.
Alpine support currently applies to the explicit register-and-run flow above.
The generic `banger image build --from-image ...` path remains Debian/systemd-
oriented and should not be treated as an Alpine image builder.
## Security ## Security
Guest VMs are single-user development sandboxes, not multi-tenant servers. Guest VMs are single-user development sandboxes, not multi-tenant
Every provisioned image is configured with: servers. Every provisioned image is configured with:
``` ```
PermitRootLogin yes PermitRootLogin yes
StrictModes no StrictModes no
``` ```
This is intentional. The host SSH key is the only authentication mechanism, The host SSH key is the only authentication mechanism, no password
no password auth is enabled, and VMs are reachable only through the host auth is enabled, and VMs are reachable only through the host bridge
bridge network (`172.16.0.0/24` by default). Do not expose the bridge network (`172.16.0.0/24` by default). Do not expose the bridge
interface or the VM guest IPs to an untrusted network. interface or guest IPs to an untrusted network.
## Notes ## Notes
- Firecracker is resolved from `PATH` by default.
- Managed image delete removes the daemon-owned artifact dir. - Managed image delete removes the daemon-owned artifact dir.
- The companion vsock helper is internal to the install/build layout, not a user-configured runtime path. - Layer blob cache for OCI pulls lives under `~/.cache/banger/oci/`.
- Image bundle cache doesn't exist — bundles are extracted directly
into the image store; re-pulls download fresh.

123
docs/image-catalog.md Normal file
View file

@ -0,0 +1,123 @@
# Image catalog
The image catalog ships pre-built banger rootfs bundles so users don't
have to register or build anything. It's the fast path behind
`banger vm run` (auto-pull) and `banger image pull <name>`. The
catalog is embedded into the banger binary and updated each release.
End-user flow:
```bash
banger image pull debian-bookworm # explicit
banger vm run --name sandbox # implicit (auto-pulls)
```
## Architecture
Two parts — the same shape as the kernel catalog:
1. **`internal/imagecat/catalog.json`** — JSON manifest embedded into
the banger binary via `go:embed`. Each entry: name, distro, arch,
kernel_ref (a `kernelcat` entry name), tarball URL, tarball
sha256, size.
2. **Tarballs at `https://images.thaloco.com/`** — Cloudflare R2
bucket `banger-images`, fronted by a public custom domain. Each
tarball is `<name>-<arch>-<sha256-prefix>.tar.zst` (content-
addressed filename so CDN edge cache can never serve stale bytes
for the URL the catalog points at). Contents at the archive root:
`rootfs.ext4` (finalized: flattened + ownership-fixed + agent-
injected at build time) and `manifest.json`.
The `banger image pull` bundle path streams the tarball, verifies
sha256 against the catalog entry, extracts both files into a staging
dir, resolves the kernel via `kernel_ref` (auto-pulling from
`kernelcat` if the user hasn't pulled it yet), stages boot artifacts
alongside, and registers the result as a managed image.
The same `image pull` command transparently falls through to the
existing OCI-pull path when `<name>` doesn't match a catalog entry —
see [`docs/oci-import.md`](oci-import.md).
## Adding or updating an entry
The repo has no CI for bundle publishing yet. Catalog updates are
manual.
```bash
# 1. Build the bundle + upload + patch catalog.json in one shot.
scripts/publish-golden-image.sh
# 2. Review and commit the catalog change.
git diff -- internal/imagecat/catalog.json
git add internal/imagecat/catalog.json
git commit -m 'imagecat: publish debian-bookworm'
# 3. Rebuild so the new catalog is embedded.
make build
```
`scripts/publish-golden-image.sh` wraps `scripts/make-golden-bundle.sh`
(which runs `docker build` on `images/golden/Dockerfile` then pipes
`docker export` into `banger internal make-bundle`), computes the
bundle's sha256, uses the first 12 hex chars as a cache-busting
filename suffix, uploads via `rclone` to R2, HEAD-checks the public
URL, and patches `internal/imagecat/catalog.json`.
Environment overrides if the defaults need to change:
`RCLONE_REMOTE`, `RCLONE_BUCKET`, `BASE_URL`.
`--skip-upload` builds the bundle into `dist/` and stops — useful for
local testing without touching R2 or the catalog.
## Bundle format
A bundle is a tar+zstd archive with exactly two entries at the root:
```
rootfs.ext4 # finalized banger rootfs
manifest.json # {name, distro, arch, kernel_ref, description}
```
`rootfs.ext4` is fully prepared at build time: ownership fixed via
`debugfs sif`, banger guest agents (vsock agent, network bootstrap,
first-boot unit) already injected and enabled in
`multi-user.target.wants`. The pull path only has to place the file
and register the image — no mkfs, no ownership pass, no injection on
the daemon host.
## Removing an entry
1. Remove the entry from `internal/imagecat/catalog.json` and commit.
2. Delete the tarball from R2:
`rclone delete banger-images:banger-images/<name>-<arch>-<hash>.tar.zst`.
3. Rebuild banger.
Already-pulled local images are not invalidated — users keep using
them until they run `banger image delete <name>`.
## Versioning conventions
- **Entry names**: `<distro>-<release>` (e.g. `debian-bookworm`).
Per-release names make it trivial to publish `debian-trixie`
alongside without collisions.
- **Content-addressed filenames**: the `-<sha256-prefix>` suffix is
mandatory (set by `publish-golden-image.sh`). Never reuse a URL for
different bytes.
- **Architecture**: `x86_64` only today. The `arch` field is additive
— adding `arm64` is a config change, not a schema change.
## Trust model
Same as the kernel catalog: the embedded `catalog.json` carries each
bundle's sha256, and `imagecat.Fetch` rejects any download whose hash
doesn't match. This protects against transport corruption and against
an attacker swapping an R2 object without landing a commit in the
banger repo. GPG/sigstore signing is deferred until banger is public
and the threat model justifies the operational overhead.
## Hosting
Tarballs live in Cloudflare R2 (bucket `banger-images`), served at
`images.thaloco.com`. The bucket is publicly readable; writes require
the R2 API token configured on the `banger-images` rclone remote.

262
docs/oci-import.md
View file

@ -1,193 +1,153 @@
# OCI import (`banger image pull`) # OCI import (`banger image pull`)
`banger image pull <oci-ref>` downloads a container image from any `banger image pull` has two paths. The primary one — catalog bundle —
OCI-compatible registry (Docker Hub, GHCR, quay.io, self-hosted, …), is documented in [`docs/image-catalog.md`](image-catalog.md). This
flattens its layers into an ext4 rootfs, and registers the result as doc covers the fallthrough: OCI-registry pull for arbitrary container
a managed banger image. images.
Paired with the kernel catalog, this dissolves the "where do I get a ## When to use it
rootfs" bottleneck for most users — any distro that ships an official
container image can now boot (eventually) as a banger VM. Use the OCI path when you need a distro or image that isn't in the
catalog. The catalog covers the common happy path
(`debian-bookworm`); anything else (`alpine`, `fedora`, `ubuntu`,
custom corporate images) goes through OCI pull.
```bash ```bash
banger kernel pull void-6.12 banger image pull docker.io/library/alpine:3.20 --kernel-ref generic-6.12
banger image pull docker.io/library/debian:bookworm --kernel-ref void-6.12 banger image pull ghcr.io/myorg/devimg:v2 --kernel-ref generic-6.12
banger image list # debian-bookworm appears, Managed=true
``` ```
`banger image pull` dispatches based on the reference:
- `banger image pull debian-bookworm` → catalog (fast path).
- `banger image pull docker.io/library/foo:bar` → OCI (anything not
in the catalog).
## What works ## What works
- Pulling any public OCI image that exposes a `linux/amd64` manifest. - Any public OCI image that exposes a `linux/amd64` manifest.
- Correct layer replay with whiteout semantics (`.wh.*` deletes, - Correct layer replay with whiteout semantics (`.wh.*` deletes,
`.wh..wh..opq` opaque-dir markers). `.wh..wh..opq` opaque-dir markers).
- Path-traversal and relative-symlink-escape protection. - Path-traversal and relative-symlink-escape protection.
- Content-aware default sizing (`content × 1.25`, floor 1 GiB). - Content-aware default sizing (`content × 1.5`, floor 1 GiB).
- Layer caching on disk, keyed by blob SHA256. - Layer caching on disk, keyed by blob sha256.
- **File ownership preservation.** Tar-header uid/gid/mode is captured - **Ownership preservation** — tar-header uid/gid/mode captured
during flatten and applied to the resulting ext4 via a `debugfs` during flatten, applied to the ext4 via a `debugfs` pass, so
pass, so setuid binaries (`sudo`, `passwd`) and root-owned config setuid binaries (`sudo`, `passwd`) and root-owned config
files (`/etc/shadow`, `/etc/sudoers`) end up correctly owned. (`/etc/shadow`, `/etc/sudoers`) end up correctly owned.
- **Banger guest agents pre-injected.** The pulled ext4 ships with - **Pre-injected banger agents** — the pulled ext4 ships with
`/usr/local/bin/banger-vsock-agent`, `banger-network.service`, and `banger-vsock-agent`, `banger-network.service`, and the
`banger-vsock-agent.service` already in place and enabled. `banger-first-boot` unit already enabled.
- **First-boot sshd install.** A one-shot systemd service installs - **First-boot sshd install** — a one-shot systemd service installs
`openssh-server` via the guest's package manager on first boot — `openssh-server` via the guest's package manager on first boot.
apt-get / apk / dnf / pacman / zypper dispatch based on Dispatches on `/etc/os-release` → `apt-get` / `apk` / `dnf` /
`/etc/os-release`. Subsequent boots skip the install. `pacman` / `zypper`. Subsequent boots skip the install.
- Piping pulled images into the existing `banger image build - Composition with `image build --from-image`.
--from-image` flow.
## What doesn't yet work ## What doesn't yet work
- **Private registries**. Auth is not implemented; anonymous pulls - **Private registries**. Anonymous pulls only. Docker Hub, GHCR
only. Docker Hub, GHCR (public), quay.io (public), etc. all work. (public), quay.io (public) all work. Adding auth via
- **Non-`linux/amd64` platforms**. The kernel catalog is x86_64-only, `authn.DefaultKeychain` (from `go-containerregistry`) is a cheap
so pulled rootfses match. `arm64` is additive in the schema; wire- follow-up when someone needs it.
up lands when a user needs it. - **Non-`linux/amd64`**. The kernel catalog is x86_64-only, so pulled
- **Non-systemd distros.** The injected units assume systemd as PID 1. rootfses match. `arm64` is additive in the schema.
Alpine ≥3.20 ships systemd; older alpine + void + busybox-init - **Non-systemd rootfses**. The injected units assume systemd as
images won't honour the banger-network / banger-first-boot units. PID 1. Alpine ≥3.20 ships systemd; older alpine + void + busybox-
- **First boot needs network access.** The provisioning step reaches init images won't honour the banger-* units.
out to the distro's package repo to install openssh-server. VMs - **First boot needs network access**. The first-boot sshd install
without NAT or without the bridge reaching the internet will time reaches out to the distro's package repo. VMs without NAT or
out on first boot. The marker file stays in place so a later boot without the bridge reaching the internet time out. The marker file
retries. stays in place so a later restart retries.
## Architecture ## Architecture
`internal/imagepull/` owns the pure mechanics: `internal/imagepull/` owns the mechanics:
- **`Pull`** (`imagepull.go`) wraps `go-containerregistry`'s - **`Pull`** wraps `go-containerregistry`'s `remote.Image` with the
`remote.Image` with the `linux/amd64` platform pinned. Layer `linux/amd64` platform pinned. Layer blobs cache under
blobs are cached on disk via `cache.NewFilesystemCache` under `~/.cache/banger/oci/blobs/` and populate lazily during flatten.
`<OCICacheDir>/blobs/` — Pull itself does not drain the layer - **`Flatten`** replays layers oldest-first into a staging directory,
streams; that happens lazily during `Flatten`, and the cache applies whiteouts, rejects unsafe paths. Returns a `Metadata` map
populates on read. of per-file uid/gid/mode from tar headers.
- **`Flatten`** (`flatten.go`) replays layers oldest-first into a - **`BuildExt4`** runs `mkfs.ext4 -F -d <staging> -E root_owner=0:0`
staging directory, applying whiteouts and rejecting unsafe paths. at the size of the pre-truncated file — no mount, no sudo, no
Returns a `Metadata` map capturing per-file uid/gid/mode from loopback. Requires `e2fsprogs ≥ 1.43`.
each tar header. - **`ApplyOwnership`** streams a batched `set_inode_field` script to
- **`BuildExt4`** (`ext4.go`) runs `mkfs.ext4 -F -d <staging> `debugfs -w` to rewrite per-file uid/gid/mode to the captured tar-
-E root_owner=0:0` to populate the image file at create time — header values.
no mount, no sudo, no loopback. Requires `e2fsprogs ≥ 1.43` - **`InjectGuestAgents`** uses the same `debugfs` scripting to drop
(`mkfs.ext4 -d` is the populate-at-create flag; nearly all banger's guest assets into the ext4 with root ownership:
modern distros ship it). vsock agent binary, network bootstrap + unit, first-boot script +
- **`ApplyOwnership`** (`ownership.go`) streams a batched unit, `multi-user.target.wants` symlinks, vsock modules-load
`set_inode_field` script to `debugfs -w -f -` to rewrite per-file config, `/var/lib/banger/first-boot-pending` marker.
uid/gid/mode to the captured tar-header values. Without this pass
the ext4 would carry the runner's on-disk uids.
- **`InjectGuestAgents`** (`inject.go`) uses the same `debugfs`
scripting to drop banger's guest-side assets into the pulled ext4
with root ownership:
- `/usr/local/bin/banger-vsock-agent`
- `/usr/local/libexec/banger-network-bootstrap`
- `/usr/local/libexec/banger-first-boot`
- `/etc/systemd/system/banger-{network,vsock-agent,first-boot}.service`
- enable-at-boot symlinks under `multi-user.target.wants/`
- `/etc/modules-load.d/banger-vsock.conf`
- `/var/lib/banger/first-boot-pending` (marker file)
`internal/daemon/images_pull.go` orchestrates: `internal/daemon/images_pull.go` orchestrates `pullFromOCI`:
1. Parse + validate the OCI ref. 1. Parse + validate the OCI ref, derive a default name when `--name`
2. Derive a friendly default name (`debian-bookworm` for is omitted (`debian-bookworm` from
`docker.io/library/debian:bookworm`) when `--name` is omitted. `docker.io/library/debian:bookworm`).
3. Resolve kernel info via the shared `resolveKernelInputs` helper 2. Resolve kernel info via `resolveKernelInputs` (auto-pulls from
(the same code path as `image register --kernel-ref`). `kernelcat` if `--kernel-ref` names a catalog entry that isn't
4. Stage at `<ImagesDir>/<id>.staging`; extract layers to a temp yet local).
tree under `os.TempDir` (bulk transient data stays off the 3. Stage at `<ImagesDir>/<id>.staging`; extract layers to a temp
persistent state filesystem). tree under `$TMPDIR`.
5. `imagepull.BuildExt4` produces `<staging>/rootfs.ext4`. 4. `BuildExt4` → `ApplyOwnership` → `InjectGuestAgents`.
6. `ApplyOwnership` + `InjectGuestAgents` run in one finalize step. 5. `imagemgr.StageBootArtifacts` stages the kernel triple alongside.
7. `imagemgr.StageBootArtifacts` stages the kernel triple alongside. 6. Atomic `os.Rename` publishes the artifact dir.
8. Atomic `os.Rename(<staging>, <final>)` publishes the artifact dir. 7. Persist a `model.Image{Managed: true, …}` record.
9. Persist a `model.Image{Managed: true, …}` record.
Any failure removes the staging dir. Post-rename failures remove the
final dir and roll back the store write.
## Guest-side boot sequence ## Guest-side boot sequence
On the first boot of a pulled image, systemd starts three banger On first boot of a pulled image:
units in order:
1. **`banger-network.service`** — runs the bootstrap script that 1. **`banger-network.service`** — brings the guest interface up with
parses `/etc/banger-network.conf` (written by banger's VM-create the IP assigned by banger's VM-create lifecycle.
lifecycle) and brings the guest interface up with the assigned IP. 2. **`banger-first-boot.service`** (first boot only) — reads
2. **`banger-first-boot.service`** (only on first boot; removes its `/etc/os-release`, dispatches to the native package manager,
own trigger file on success) — reads `/etc/os-release`, dispatches installs `openssh-server`, enables `ssh.service`.
to the native package manager, installs `openssh-server`, enables 3. **`banger-vsock-agent.service`** — the health-check daemon banger
`ssh.service` / `sshd.service`. uses to confirm the VM is alive.
3. **`banger-vsock-agent.service`** — runs the health-check daemon
banger uses to confirm the VM is alive.
After first boot completes, subsequent boots skip the install step Subsequent boots skip step 2.
entirely. Banger's host-side SSH polling (`guest.WaitForSSH`)
naturally retries until sshd is listening.
## Adding distro support ## Adding distro support to first-boot
`internal/imagepull/assets/first-boot.sh` is the POSIX-sh dispatch. `internal/imagepull/assets/first-boot.sh` is the POSIX-sh dispatch.
Add a new `ID=` branch and its install command to the `case` block, Add a new `ID=` branch and its install command, then rebuild banger
then rebuild banger — the asset is `go:embed`-ed into the binary. (the asset is `go:embed`-ed).
Supported `ID` values today: `debian`, `ubuntu`, `kali`, `raspbian`, Supported `ID` values today: `debian`, `ubuntu`, `kali`, `raspbian`,
`linuxmint`, `pop`, `alpine`, `fedora`, `rhel`, `centos`, `rocky`, `linuxmint`, `pop`, `alpine`, `fedora`, `rhel`, `centos`, `rocky`,
`almalinux`, `arch`, `archlinux`, `manjaro`, `opensuse*`, `suse`. `almalinux`, `arch`, `archlinux`, `manjaro`, `opensuse*`, `suse`.
Unknown distros fall back to `ID_LIKE`, then error clearly with a Unknown distros fall back to `ID_LIKE`, then error cleanly.
pointer to edit the script.
## Paths ## Paths
| What | Where | Purpose | | What | Where |
|------|-------|---------| |------|-------|
| Layer blob cache | `~/.cache/banger/oci/blobs/sha256/<hex>` | Re-pulls of the same image digest are local-only | | Layer blob cache | `~/.cache/banger/oci/blobs/sha256/<hex>` |
| Staging dir | `~/.local/state/banger/images/<id>.staging/` | Short-lived; atomic-renamed to `<id>/` on success | | Staging dir | `~/.local/state/banger/images/<id>.staging/` |
| Staging rootfs tree | `$TMPDIR/banger-pull-<rand>/` | Extraction scratch space; removed after ext4 build | | Extraction scratch | `$TMPDIR/banger-pull-<rand>/` |
| Published image | `~/.local/state/banger/images/<id>/rootfs.ext4` | Managed artifact stored alongside the kernel triple | | Published image | `~/.local/state/banger/images/<id>/rootfs.ext4` |
## Composition with `image build`
A pulled image boots as-is — ownership is correct, sshd installs on
first boot, banger's agents are in place. That means the existing
`image build --from-image` pipeline composes on top:
```bash
banger image build --from-image debian-bookworm --name debian-dev --docker
```
`image build` spins up a transient VM using the base image, runs
`scripts/customize.sh` over it, and saves the result as a new managed
image with the opinionated tooling (mise, opencode, claude, pi, tmux
plugins, optionally docker) layered on top.
## Tech debt ## Tech debt
- **Auth**. When we add private-registry support, the natural path is - **Auth**. When we add private-registry support, the natural path
`authn.DefaultKeychain` from `go-containerregistry`, which already is `authn.DefaultKeychain`, which honours `~/.docker/config.json`
honours `~/.docker/config.json` and the standard credential and the standard credential helpers.
helpers. No banger-specific config needed. - **Cache eviction**. OCI layer blobs accumulate forever. A `banger
image cache prune` command is a cheap follow-up when disk usage
- **Cache eviction**. Layer blobs under `OCICacheDir` accumulate becomes a complaint.
forever. A `banger image cache prune` command is a cheap follow-up - **Non-systemd rootfses**. The guest agents assume systemd. Adding
when disk usage becomes a complaint.
- **First-boot timeout UX**. If you run `banger vm ssh` immediately
after `banger vm create`, the package install for `openssh-server`
may still be running and SSH will fail. Current mitigation: retry.
Better: a per-image `FirstBootPending` flag that tells the daemon
to extend its SSH wait timeout for the first boot, cleared on
success. Tracked but not implemented.
- **Non-systemd distros**. The guest agents assume systemd. Adding
openrc / s6 / busybox-init variants means keeping parallel unit openrc / s6 / busybox-init variants means keeping parallel unit
trees in `inject.go` keyed on `/etc/os-release`. Only pick up trees keyed on `/etc/os-release`.
when a user actually wants it.
## Trust model ## Trust model
`image pull` delegates trust to the OCI registry the user selected. `image pull` (OCI path) delegates trust to the registry the user
`go-containerregistry` verifies layer digests against the manifest selected. `go-containerregistry` verifies layer digests against the
during download, so a tampered mirror can't ship modified layers manifest during download, so a tampered mirror can't ship modified
without breaking the sha256 chain. Beyond that, banger does not layers without breaking the sha256 chain. Banger does not verify OCI
verify OCI image signatures (cosign/sigstore) — users who care should image signatures (cosign/sigstore) — users who care should verify
verify their references out-of-band. references out-of-band.