diff --git a/AGENTS.md b/AGENTS.md index 5cfea46..60d086d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,55 +1,49 @@ # Repository Guidelines -## Project Structure & Module Organization -- `cmd/banger` and `cmd/bangerd` are the primary user-facing entrypoints. -- `internal/` contains the daemon, CLI, RPC, storage, Firecracker, and system integration code. -- The VM lifecycle is now organized around daemon capabilities plus a structured guest-config builder. New host-integrated VM features should plug into that Go path instead of adding more one-off branches through `internal/daemon/vm.go`. -- `scripts/customize.sh`, `scripts/make-rootfs.sh`, and `scripts/interactive.sh` remain as manual rootfs/customization helpers; normal VM lifecycle, NAT, `.vm` DNS, and daemon-driven image builds are handled by the Go control plane. -- Source checkouts use a generated `./build/runtime/` bundle for Firecracker, kernels, modules, rootfs images, and helper copies. Bundle defaults come from `./build/runtime/bundle.json` when present. Those runtime artifacts are not meant to be tracked directly in Git. -- The daemon keeps state under XDG directories rather than the old repo-local `state/` layout. +## Project Structure -## Build, Test, and Development Commands -- `make build` builds `./build/bin/banger`, `./build/bin/bangerd`, and the bundled `./build/runtime/banger-vsock-agent` guest helper. -- `make bench-create` benchmarks `vm create` and first-SSH readiness on the current host. -- `make runtime-bundle` bootstraps `./build/runtime/` from the archive referenced by `RUNTIME_MANIFEST`; the checked-in `config/runtime-bundle.toml` is only a template. -- `make void-kernel` downloads and stages a Void `linux6.12` kernel under `./build/runtime/void-kernel`, including extracted `vmlinux`, raw `vmlinuz`, a matching generated `initramfs`, config, and matching modules. -- `make rootfs-void` builds an experimental local-only `x86_64-glibc` Void rootfs plus work-seed under `./build/runtime/`; it prefers staged `./build/runtime/void-kernel` modules when present, but does not replace the default Debian path or teach `banger image build` about Void. -- `make verify-void` registers `void-exp` and runs the normal smoke test against that image. -- `banger` validates required host tools per command and reports actionable missing-tool errors; do not assume one workstation's package set. -- `./build/bin/banger vm create --name testbox` creates and starts a VM. -- `./build/bin/banger vm create` now blocks until the guest reaches the daemon's default readiness checks and shows live progress stages on TTY stderr while it waits. -- `./build/bin/banger vm ssh testbox` connects to a running guest using the runtime bundle SSH key and reminds the user if the VM is still running when the session exits. -- `./build/bin/banger vm stop testbox` stops a VM while preserving its disks. -- `./build/bin/banger vm stop vm-a vm-b vm-c` and `./build/bin/banger vm set --nat web-1 web-2` are supported; multi-VM lifecycle and `set` actions fan out concurrently through the CLI. -- `./build/bin/banger doctor` reports runtime bundle, host tool, feature, and image-build readiness from the same Go checks used by the daemon. -- `./build/bin/banger image register --name local --rootfs /abs/path/rootfs.ext4` creates or updates an unmanaged image record without changing the default image config; use it for experimental guest iteration paths such as Void. -- `bangerd` now also serves a localhost web UI on `http://127.0.0.1:7777` by default unless `web_listen_addr = ""` disables it; the UI uses server-rendered templates, polls async VM/image operations, and keeps image path selection on the host via a server-side file picker. +- `cmd/banger` and `cmd/bangerd` are the main user entrypoints. +- `internal/` contains the daemon, CLI, RPC, storage, Firecracker integration, guest helpers, and web UI. +- `scripts/` contains explicit manual helper workflows for rootfs and kernel preparation. +- `build/bin/` is the canonical source-checkout build output. +- `build/manual/` is the canonical source-checkout location for manual rootfs/kernel artifacts. + +## Build and Test + +- `make build` builds `./build/bin/banger`, `./build/bin/bangerd`, and `./build/bin/banger-vsock-agent`. - `make test` runs `go test ./...`. -- `./scripts/verify.sh` runs the smoke test for the Go VM workflow. +- `./build/bin/banger doctor` checks host readiness. +- `./build/bin/banger image build --from-image ` builds a managed image from an existing registered image. +- `./build/bin/banger image register ...` registers an unmanaged host-side image stack. +- `./build/bin/banger image promote ` copies an unmanaged image into daemon-owned managed artifacts. +- `make void-kernel`, `make rootfs-void`, and `make void-register` drive the experimental Void flow under `./build/manual`. -## Coding Style & Naming Conventions -- Go code should stay small, direct, and standard-library-first unless there is a clear reason otherwise. -- Shell helpers use Bash with `set -euo pipefail`; keep remaining shell scripts strict and explicit. -- Prefer lowercase filenames with short descriptive names. -- Use `gofmt` for Go formatting; no extra formatter is configured for shell files. +## Image Model + +- Managed images own the full boot set: rootfs, optional work-seed, kernel, optional initrd, and optional modules. +- There is no runtime bundle and no auto-registered default image from disk paths. +- `default_image_name` selects a registered image only. + +## Config + +- Config lives at `~/.config/banger/config.toml`. +- Firecracker comes from `PATH` by default, or `firecracker_bin`. +- SSH uses `ssh_key_path` or an auto-managed default key at `~/.config/banger/ssh/id_ed25519`. + +## Coding Style + +- Prefer small, direct Go code and standard library solutions. +- Keep shell scripts strict with `set -euo pipefail`. +- Use `gofmt` for Go formatting. + +## Testing Guidance -## Testing Guidelines - Primary automated coverage is `go test ./...`. -- Manual verification for VM lifecycle changes: `./build/bin/banger vm create`, confirm SSH access, then stop/delete the VM. -- For host-integration changes, run `./build/bin/banger doctor` as a quick readiness check before the live VM smoke. -- The web UI follows the same sudo model as the CLI path: bangerd stays unprivileged and privileged writes only work when `sudo -v` is already warm or sudo is passwordless. -- Rebuilt images now include `mise`, `opencode`, a host-reachable default `opencode` server service on guest TCP port `4096`, `tmux-resurrect`/`tmux-continuum` defaults for `root`, and the `banger-vsock-agent` service used by the SSH reminder and guest health-check path; if you change guest provisioning, document whether users need to rebuild `./build/runtime/rootfs-docker.ext4` or another base image to pick it up. -- The experimental Void rootfs path now includes the repo's basic dev baseline plus Docker and Compose, alongside boot, SSH, a guest network bootstrap sourced from the kernel `ip=` cmdline, the vsock HTTP health agent, pinned `mise` plus `opencode` for `root`, the default host-reachable `opencode` server service on guest TCP port `4096`, a `bash` root shell while leaving `/bin/sh` alone, and the `/root` work-seed. When `./build/runtime/void-kernel/` exists, the Void image registration path expects a complete staged Void kernel, initramfs, and modules tree and points `void-exp` at it. Keep further baked-in tooling deliberate and user-driven. -- Rebuilt images also emit a `work-seed.ext4` sidecar used to speed up future VM creates. Older managed images may take one slower create to refresh seeded SSH access before they rejoin the fast path. If you touch `/root` provisioning, verify both the rootfs and the work-seed output. -- The daemon may keep idle TAP devices in a pool for faster creates. Smoke tests should treat `tap-pool-*` devices as reusable capacity, not cleanup leaks. -- If you add a new operational workflow, document how to exercise it in `README.md`. -- For NAT changes, verify both guest outbound access and host rule cleanup, for example with `./scripts/verify.sh --nat`. +- For lifecycle changes, smoke-test with `vm create`, `vm ssh`, `vm stop`, and `vm delete`. +- If guest provisioning changes, document whether existing images must be rebuilt or recreated. -## Commit & Pull Request Guidelines -- Git history uses short, imperative subjects. -- Prefer a real commit body when the change affects lifecycle behavior, storage semantics, or host integration. -- PRs should call out runtime requirements, migration impact, and any host-side verification performed. +## Security -## Security & Configuration Tips -- The VM workflow requires `sudo` and `/dev/kvm` access; do not commit secrets. -- `id_ed25519` lives inside the runtime bundle; rotate or replace it before publishing a shared bundle. +- Do not commit secrets. +- VM workflows require `sudo` and `/dev/kvm`. +- The default SSH key is local configuration, not a checked-in runtime artifact. diff --git a/Makefile b/Makefile index 50a17fa..1b4e37e 100644 --- a/Makefile +++ b/Makefile @@ -6,50 +6,40 @@ INSTALL ?= install PREFIX ?= $(HOME)/.local BINDIR ?= $(PREFIX)/bin LIBDIR ?= $(PREFIX)/lib -RUNTIMEDIR ?= $(LIBDIR)/banger DESTDIR ?= BUILD_DIR ?= build BUILD_BIN_DIR ?= $(BUILD_DIR)/bin -RUNTIME_MANIFEST ?= config/runtime-bundle.toml -RUNTIME_SOURCE_DIR ?= $(BUILD_DIR)/runtime -RUNTIME_ARCHIVE ?= $(BUILD_DIR)/dist/banger-runtime.tar.gz +BUILD_MANUAL_DIR ?= $(BUILD_DIR)/manual BANGER_BIN ?= $(BUILD_BIN_DIR)/banger BANGERD_BIN ?= $(BUILD_BIN_DIR)/bangerd -BINARIES := $(BANGER_BIN) $(BANGERD_BIN) -RUNTIME_HELPERS := $(RUNTIME_SOURCE_DIR)/banger-vsock-agent +VSOCK_AGENT_BIN ?= $(BUILD_BIN_DIR)/banger-vsock-agent +BINARIES := $(BANGER_BIN) $(BANGERD_BIN) $(VSOCK_AGENT_BIN) GO_SOURCES := $(shell find cmd internal -type f -name '*.go' | sort) -RUNTIME_EXECUTABLES := firecracker customize.sh packages.sh namegen banger-vsock-agent -RUNTIME_DATA_FILES := packages.apt id_ed25519 rootfs-docker.ext4 -RUNTIME_OPTIONAL_DATA_FILES := rootfs.ext4 rootfs-docker.work-seed.ext4 bundle.json -RUNTIME_BOOT_FILES := wtf/root/boot/vmlinux-6.8.0-94-generic wtf/root/boot/initrd.img-6.8.0-94-generic -RUNTIME_MODULES_DIR := wtf/root/lib/modules/6.8.0-94-generic VOID_IMAGE_NAME ?= void-exp VOID_VM_NAME ?= void-dev .DEFAULT_GOAL := help -.PHONY: help build banger bangerd test fmt tidy clean rootfs rootfs-void void-kernel void-register void-vm verify-void install runtime-bundle runtime-package check-runtime bench-create +.PHONY: help build banger bangerd test fmt tidy clean rootfs rootfs-void void-kernel void-register void-vm verify-void install bench-create help: @printf '%s\n' \ 'Targets:' \ - ' make build Build ./build/bin/banger and ./build/bin/bangerd' \ - ' make runtime-bundle Fetch and unpack ./build/runtime from the archive referenced by $(RUNTIME_MANIFEST)' \ - ' make runtime-package Package $(RUNTIME_SOURCE_DIR) into $(RUNTIME_ARCHIVE) and print its SHA256' \ + ' make build Build ./build/bin/banger, ./build/bin/bangerd, and ./build/bin/banger-vsock-agent' \ ' make bench-create Benchmark vm create and SSH readiness with scripts/bench-create.sh' \ - ' make install Build and install binaries plus the runtime bundle into $(DESTDIR)$(BINDIR) and $(DESTDIR)$(RUNTIMEDIR)' \ + ' make install Build and install banger, bangerd, and the companion vsock helper' \ ' make test Run go test ./...' \ ' make fmt Format Go sources under cmd/ and internal/' \ ' make tidy Run go mod tidy' \ ' make clean Remove built Go binaries' \ - ' make rootfs Rebuild the source-checkout default Debian rootfs image in ./build/runtime' \ - ' make void-kernel Download and stage a Void kernel, initramfs, and modules under ./build/runtime/void-kernel' \ - ' make rootfs-void Build an experimental Void Linux rootfs and work-seed in ./build/runtime' \ + ' make rootfs Rebuild the manual Debian rootfs image in ./build/manual' \ + ' make void-kernel Download and stage a Void kernel, initramfs, and modules under ./build/manual/void-kernel' \ + ' make rootfs-void Build an experimental Void Linux rootfs and work-seed in ./build/manual' \ ' make void-register Register or update the experimental Void image as $(VOID_IMAGE_NAME)' \ ' make void-vm Register the experimental Void image and create a VM named $(VOID_VM_NAME)' \ ' make verify-void Register the experimental Void image and run scripts/verify.sh against it' -build: $(BINARIES) $(RUNTIME_HELPERS) +build: $(BINARIES) $(BANGER_BIN): $(GO_SOURCES) go.mod go.sum mkdir -p "$(BUILD_BIN_DIR)" @@ -59,9 +49,9 @@ $(BANGERD_BIN): $(GO_SOURCES) go.mod go.sum mkdir -p "$(BUILD_BIN_DIR)" $(GO) build -o "$(BANGERD_BIN)" ./cmd/bangerd -$(RUNTIME_SOURCE_DIR)/banger-vsock-agent: $(GO_SOURCES) go.mod go.sum - mkdir -p "$(RUNTIME_SOURCE_DIR)" - CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GO) build -o "$(RUNTIME_SOURCE_DIR)/banger-vsock-agent" ./cmd/banger-vsock-agent +$(VSOCK_AGENT_BIN): $(GO_SOURCES) go.mod go.sum + mkdir -p "$(BUILD_BIN_DIR)" + CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GO) build -o "$(VSOCK_AGENT_BIN)" ./cmd/banger-vsock-agent test: $(GO) test ./... @@ -74,55 +64,28 @@ tidy: clean: rm -rf "$(BUILD_BIN_DIR)" - rm -f "$(RUNTIME_SOURCE_DIR)/banger-vsock-agent" - -runtime-bundle: - $(GO) run ./cmd/runtimebundle fetch --manifest "$(RUNTIME_MANIFEST)" --out "$(RUNTIME_SOURCE_DIR)" - -runtime-package: - $(GO) run ./cmd/runtimebundle package --manifest "$(RUNTIME_MANIFEST)" --runtime-dir "$(RUNTIME_SOURCE_DIR)" --out "$(RUNTIME_ARCHIVE)" bench-create: build BANGER_BIN="$(abspath $(BANGER_BIN))" bash ./scripts/bench-create.sh $(ARGS) -check-runtime: - @test -d "$(RUNTIME_SOURCE_DIR)" || { echo "missing runtime bundle directory: $(RUNTIME_SOURCE_DIR); run 'make runtime-bundle'" >&2; exit 1; } - @for path in $(RUNTIME_EXECUTABLES) $(RUNTIME_DATA_FILES) $(RUNTIME_BOOT_FILES) $(RUNTIME_MODULES_DIR); do \ - test -e "$(RUNTIME_SOURCE_DIR)/$$path" || { echo "missing runtime artifact: $(RUNTIME_SOURCE_DIR)/$$path; run 'make runtime-bundle'" >&2; exit 1; }; \ - done - -install: build check-runtime +install: build mkdir -p "$(DESTDIR)$(BINDIR)" - mkdir -p "$(DESTDIR)$(RUNTIMEDIR)" - mkdir -p "$(DESTDIR)$(RUNTIMEDIR)/wtf/root/boot" - mkdir -p "$(DESTDIR)$(RUNTIMEDIR)/wtf/root/lib/modules" + mkdir -p "$(DESTDIR)$(LIBDIR)/banger" $(INSTALL) -m 0755 "$(BANGER_BIN)" "$(DESTDIR)$(BINDIR)/banger" $(INSTALL) -m 0755 "$(BANGERD_BIN)" "$(DESTDIR)$(BINDIR)/bangerd" - @for path in $(RUNTIME_EXECUTABLES); do \ - $(INSTALL) -m 0755 "$(RUNTIME_SOURCE_DIR)/$$path" "$(DESTDIR)$(RUNTIMEDIR)/$$path"; \ - done - @for path in $(RUNTIME_DATA_FILES) $(RUNTIME_BOOT_FILES); do \ - $(INSTALL) -m 0644 "$(RUNTIME_SOURCE_DIR)/$$path" "$(DESTDIR)$(RUNTIMEDIR)/$$path"; \ - done - @for path in $(RUNTIME_OPTIONAL_DATA_FILES); do \ - if test -e "$(RUNTIME_SOURCE_DIR)/$$path"; then \ - $(INSTALL) -m 0644 "$(RUNTIME_SOURCE_DIR)/$$path" "$(DESTDIR)$(RUNTIMEDIR)/$$path"; \ - fi; \ - done - chmod 0600 "$(DESTDIR)$(RUNTIMEDIR)/id_ed25519" - cp -a "$(RUNTIME_SOURCE_DIR)/$(RUNTIME_MODULES_DIR)" "$(DESTDIR)$(RUNTIMEDIR)/wtf/root/lib/modules/" + $(INSTALL) -m 0755 "$(VSOCK_AGENT_BIN)" "$(DESTDIR)$(LIBDIR)/banger/banger-vsock-agent" rootfs: - BANGER_RUNTIME_DIR="$(abspath $(RUNTIME_SOURCE_DIR))" ./scripts/make-rootfs.sh + BANGER_MANUAL_DIR="$(abspath $(BUILD_MANUAL_DIR))" BANGER_BIN="$(abspath $(BANGER_BIN))" ./scripts/make-rootfs.sh $(ARGS) void-kernel: - BANGER_RUNTIME_DIR="$(abspath $(RUNTIME_SOURCE_DIR))" ./scripts/make-void-kernel.sh + BANGER_MANUAL_DIR="$(abspath $(BUILD_MANUAL_DIR))" ./scripts/make-void-kernel.sh $(ARGS) rootfs-void: - BANGER_RUNTIME_DIR="$(abspath $(RUNTIME_SOURCE_DIR))" ./scripts/make-rootfs-void.sh + BANGER_MANUAL_DIR="$(abspath $(BUILD_MANUAL_DIR))" BANGER_BIN="$(abspath $(BANGER_BIN))" ./scripts/make-rootfs-void.sh $(ARGS) void-register: build - BANGER_RUNTIME_DIR="$(abspath $(RUNTIME_SOURCE_DIR))" VOID_IMAGE_NAME="$(VOID_IMAGE_NAME)" BANGER_BIN="$(abspath $(BANGER_BIN))" ./scripts/register-void-image.sh + BANGER_MANUAL_DIR="$(abspath $(BUILD_MANUAL_DIR))" VOID_IMAGE_NAME="$(VOID_IMAGE_NAME)" BANGER_BIN="$(abspath $(BANGER_BIN))" ./scripts/register-void-image.sh void-vm: void-register "$(abspath $(BANGER_BIN))" vm create --image "$(VOID_IMAGE_NAME)" --name "$(VOID_VM_NAME)" diff --git a/README.md b/README.md index b6b9235..88f8914 100644 --- a/README.md +++ b/README.md @@ -1,520 +1,196 @@ # banger -Persistent Firecracker development VMs managed through a Go daemon and CLI. +`banger` manages Firecracker development VMs with a local daemon, managed image artifacts, and a localhost web UI. ## Requirements -- Linux host with KVM (`/dev/kvm` access) -- Vsock support for post-SSH liveness reminders (`/dev/vhost-vsock`) -- Core VM lifecycle: `sudo`, `ip`, `dmsetup`, `losetup`, `blockdev`, `truncate`, `pgrep`, `chown`, `chmod`, `kill` -- Guest rootfs patching: `e2cp`, `e2rm`, `debugfs` -- Guest work disk creation/resizing: `mkfs.ext4`, `e2fsck`, `resize2fs`, `mount`, `umount`, `cp` -- SSH and logs: `ssh` -- Optional NAT: `iptables`, `sysctl` -- Image build: the bundled SSH key plus the tools above; `banger image build` no longer shells out through `customize.sh` -`banger` validates these per command and returns actionable errors instead of -assuming one workstation layout. +- Linux with `/dev/kvm` +- `sudo` +- Firecracker installed on `PATH`, or `firecracker_bin` set in config +- The usual host tools checked by `./build/bin/banger doctor` -## Runtime Bundle -Runtime artifacts are no longer tracked directly in Git. Source checkouts use a -generated `./build/runtime/` bundle, while installed binaries use -`$(prefix)/lib/banger`. +`banger` now owns complete managed image sets. A managed image includes: -The bundle contains: -- `firecracker` -- `banger-vsock-agent` for the guest-side vsock HTTP health agent and SSH reminder checks -- `bundle.json` with the bundle's default kernel/initrd/modules/rootfs paths -- a kernel, initrd, and modules tree referenced by `bundle.json` -- `rootfs-docker.ext4` -- `rootfs-docker.work-seed.ext4` when present, used to seed `/root` quickly on - new VM creates -- `rootfs.ext4` when present -- `packages.apt` -- `id_ed25519` -- the helper scripts used by manual customization and installs +- `rootfs` +- optional `work-seed` +- `kernel` +- optional `initrd` +- optional `modules` -Bootstrap a source checkout from a local or published runtime archive. The -checked-in [`config/runtime-bundle.toml`](/home/thales/projects/personal/banger/config/runtime-bundle.toml) -is a template and intentionally ships with empty `url` and `sha256`. - -If you need to create a local archive first, do that from a checkout or machine -that already has a populated `./build/runtime/` tree: -```bash -make runtime-package -cp build/dist/banger-runtime.tar.gz /path/to/fresh-checkout/build/dist/ -``` - -In the fresh checkout: -```bash -cp config/runtime-bundle.toml config/runtime-bundle.local.toml -``` - -Edit `config/runtime-bundle.local.toml` to point at the staged archive and checksum: -```toml -url = "./build/dist/banger-runtime.tar.gz" -sha256 = "" -``` - -Then bootstrap `./build/runtime/` with the local manifest copy: -```bash -make runtime-bundle RUNTIME_MANIFEST=config/runtime-bundle.local.toml -``` - -`url` may be a relative path, absolute path, `file:///...` URL, or HTTP(S) -URL. `make install` will not fetch artifacts for you. +There is no runtime bundle anymore. ## Build + ```bash make build ``` -Run `make build` after `./build/runtime/` has been bootstrapped. It writes -`./build/bin/banger`, `./build/bin/bangerd`, and refreshes the bundled -`banger-vsock-agent` guest helper in `./build/runtime/`. +This writes: -Older ignored root artifacts such as `./runtime/`, `./banger`, and `./bangerd` -are no longer the canonical source-checkout layout. Leave them alone if you -still need them, or remove them manually after migrating to `build/`. +- `./build/bin/banger` +- `./build/bin/bangerd` +- `./build/bin/banger-vsock-agent` -If you have confirmed your current images and runtime settings no longer point -at the old checkout-local paths, a one-time cleanup looks like: -```bash -rm -rf ./runtime ./banger ./bangerd -``` +## Install -Install into `~/.local/bin` by default, with the runtime bundle under -`~/.local/lib/banger`: ```bash make install ``` -After `make install`, the installed `banger` and `bangerd` do not need the repo -checkout to keep working. +That installs: -## Basic VM Workflow -Create and boot a VM: -```bash -banger vm create --name calm-otter --disk-size 16G -``` +- `banger` +- `bangerd` +- the `banger-vsock-agent` companion helper under `../lib/banger/` -`banger vm create` now waits for full guest readiness by default, including the -guest vsock agent and the default `opencode` service, and prints live progress -stages on TTY stderr while it waits. +## Config -Check host/runtime readiness before creating VMs: -```bash -banger doctor -``` +Config lives at `~/.config/banger/config.toml`. -List VMs: -```bash -banger vm list -``` +Supported keys: -Inspect a VM: -```bash -banger vm show calm-otter -banger vm stats calm-otter -``` - -SSH into a running VM: -```bash -banger vm ssh calm-otter -``` - -When the SSH session exits normally, `banger` checks the guest over vsock and -reminds you if the VM is still running. - -Inspect host-reachable listening ports for a running VM: -```bash -banger vm ports calm-otter -``` - -Stop, restart, kill, or delete it: -```bash -banger vm stop calm-otter -banger vm start calm-otter -banger vm restart calm-otter -banger vm kill --signal TERM calm-otter -banger vm delete calm-otter -``` - -Update stopped VM settings: -```bash -banger vm set calm-otter --memory 2048 --vcpu 4 --disk-size 32G -``` - -Lifecycle and `set` actions also accept multiple VM refs and run them -concurrently: -```bash -banger vm stop calm-otter buildbox api-1 -banger vm kill --signal KILL aa12bb34 cc56dd78 -banger vm set --nat web-1 web-2 web-3 -``` - -## Daemon -The CLI auto-starts `bangerd` when needed. - -Useful daemon commands: -```bash -banger daemon status -banger daemon socket -banger daemon stop -``` - -`banger daemon status` prints the daemon PID, socket path, daemon log path, and -the built-in DNS listener address. The daemon also serves a local web UI on -`http://127.0.0.1:7777` by default, and `daemon status` prints that URL when it -is enabled. - -Use the web UI for dashboard, VM lifecycle, image inventory, VM create -progress, ports/log inspection, and image build/register/promote/delete flows: -```text -http://127.0.0.1:7777 -``` - -The image forms use a server-side host-path picker. They do not upload files -through the browser; they select absolute paths that already exist on the host. -Mutating actions in the UI require the same sudo readiness as the CLI-backed -workflow. If the page shows writes as disabled, run: -```bash -sudo -v -``` -and refresh the page. - -State lives under XDG directories: -- config: `~/.config/banger` -- state: `~/.local/state/banger` -- cache: `~/.cache/banger` -- runtime socket: `$XDG_RUNTIME_DIR/banger/bangerd.sock` - -Installed binaries resolve their runtime bundle from `../lib/banger` relative to -the executable. Source-checkout binaries resolve it from `./build/runtime` next -to `./build/bin/banger`, and still fall back to a legacy `./runtime` checkout -bundle if that exists. You can override either with `runtime_dir` in -`~/.config/banger/config.toml` or `BANGER_RUNTIME_DIR`. - -Useful config keys: - `log_level` -- `runtime_dir` -- `web_listen_addr` (`""` disables the web UI) -- `tap_pool_size` +- `web_listen_addr` - `firecracker_bin` -- `namegen_path` -- `customize_script` (manual helper compatibility; `banger image build` is Go-native) -- `vsock_agent_path` -- `default_rootfs` -- `default_work_seed` -- `default_base_rootfs` -- `default_kernel` -- `default_initrd` -- `default_modules_dir` -- `default_packages_file` +- `ssh_key_path` +- `default_image_name` +- `auto_stop_stale_after` +- `stats_poll_interval` +- `metrics_poll_interval` +- `bridge_name` +- `bridge_ip` +- `cidr` +- `tap_pool_size` +- `default_dns` -Guest SSH access always uses the private key shipped in the resolved runtime -bundle. `ssh_key_path` is no longer a supported override for `banger vm ssh`, -VM start key injection, or daemon guest provisioning. +If `ssh_key_path` is unset, banger creates and uses: -## Doctor -`banger doctor` runs the same readiness checks the Go control plane uses for VM -start, host-integrated features, and image builds. It reports runtime bundle -state, core VM host tools, current feature readiness, and image-build -prerequisites in a concise pass/warn/fail list. +- `~/.config/banger/ssh/id_ed25519` -Use it when bringing up a new machine, after changing the runtime bundle, or -before adding new host-integrated VM features. +`default_image_name` now only means “use this registered image when `vm create` omits `--image`”. The daemon does not auto-register images from host paths. -## Logs -- daemon lifecycle logs: `~/.local/state/banger/bangerd.log` -- raw Firecracker output per VM: `~/.local/state/banger/vms//firecracker.log` -- raw image-build helper output: `~/.local/state/banger/image-build/*.log` +## Core Workflow -`bangerd.log` is structured JSON. Set `log_level` in -`~/.config/banger/config.toml` or `BANGER_LOG_LEVEL` to one of `debug`, -`info`, `warn`, or `error`. +Check the host: -## Images -List images: ```bash -banger image list +./build/bin/banger doctor ``` -Build a managed image: +Register an existing host-side image stack: + ```bash -banger image build --name docker-dev --docker +./build/bin/banger image register \ + --name base \ + --rootfs /abs/path/rootfs.ext4 \ + --kernel /abs/path/vmlinux \ + --initrd /abs/path/initrd.img \ + --modules /abs/path/modules ``` -The web UI exposes both managed image build and unmanaged image register forms. -Builds run through an async progress page; register, promote, and delete remain -direct form actions. +Build a managed image from an existing registered image: -Rebuilt images install a pinned `mise` at `/usr/local/bin/mise`, activate it -for bash login and interactive shells, install `opencode` through `mise`, -expose `/usr/local/bin/opencode`, configure `tmux-resurrect` plus -`tmux-continuum` for `root` with periodic autosaves and manual-only restore by -default, start a host-reachable `opencode serve` service on guest TCP port -`4096`, and bake in the `banger-vsock-agent` systemd service used by the -post-SSH reminder path and guest health checks. They -also emit a `work-seed.ext4` sidecar that lets new VMs clone a prepared `/root` -work disk instead of rebuilding it from scratch on every create. - -Show or delete images: ```bash -banger image show docker-dev -banger image delete docker-dev +./build/bin/banger image build \ + --name devbox \ + --from-image base \ + --docker ``` -Promote an existing unmanaged image into a managed one: +Promote an unmanaged image into daemon-owned managed artifacts: + ```bash -banger image promote default -banger image promote void-exp +./build/bin/banger image promote base ``` -Promotion copies the image's `rootfs` and optional `work-seed` into the -daemon's managed image state directory and keeps the same image ID, so existing -VM references stay valid. The image's kernel, initrd, modules, and package -manifest paths stay pointed at their current locations. +Create and use a VM: -`banger` auto-registers the bundled `default_rootfs` image when it exists. If -the bundle does not include a separate base `rootfs.ext4`, `image build` falls -back to using `rootfs-docker.ext4` as its default base image. - -## Networking And DNS -Enable NAT when creating or updating a VM: ```bash -banger vm create --name web --nat -banger vm set web --nat -banger vm set web --no-nat +./build/bin/banger vm create --image devbox --name testbox +./build/bin/banger vm ssh testbox +./build/bin/banger vm stop testbox ``` -NAT is applied by the Go control plane using host `iptables` rules derived from -the VM's current guest IP and TAP device. The remaining shell helpers also -route NAT changes through `banger` instead of a standalone shell NAT script. +`vm create` stays synchronous by default, but on a TTY it now shows live progress until the VM is fully ready. -`bangerd` also serves a tiny authoritative DNS service on `127.0.0.1:42069` -for daemon-managed VMs. Known `A` records resolve `.vm` to the VM's -guest IPv4 address. Integrate your local resolver separately if you want -transparent `.vm` lookups on the host. +## Web UI -`banger vm ports` asks the guest-side `banger-vsock-agent` to run `ss`, then -prints host-usable endpoints plus the owning process/command. TCP listeners get -short best-effort HTTP and HTTPS probes; detected web listeners are shown as -`http` or `https`, and the endpoint column becomes a clickable URL such as -`https://.vm:port/`. Older images without `ss` may need rebuilding -before `vm ports` works. +`bangerd` serves a local web UI by default at: -Newly rebuilt images also start `opencode serve` by default on guest TCP port -`4096`, bound on guest interfaces so the host can reach it directly at the -guest IP or via the endpoint shown by `banger vm ports`. +- `http://127.0.0.1:7777` -## Storage Model -- VMs share a read-only base rootfs image. -- Each VM gets its own sparse writable system overlay for `/`. -- Each VM gets its own persistent ext4 work disk mounted at `/root`. -- When an image has a `work-seed.ext4` sidecar, new VM creates clone that seed - and only resize it when needed. -- Older managed images without the seeded SSH metadata may take one slower - create to repair `/root` access and refresh their managed work-seed; later - creates use the fast path. -- Images without any `work-seed.ext4` still work, but create more slowly - because `/root` must be built from scratch. -- The daemon can keep a small idle TAP pool warm in the background so VM create - does not need to synchronously create a fresh TAP every time. `tap_pool_size` - controls the pool depth. +See the effective URL with: -## Architecture Notes -The Go daemon is the primary control plane. VM host integrations such as the -built-in `.vm` DNS service, NAT, and `/root` work-disk wiring now sit behind a -capability pipeline in the daemon instead of being open-coded through the VM -lifecycle. Guest boot-time files and mounts are rendered through a structured -guest-config builder rather than ad hoc `fstab` string mutation. - -That split is intentional: future host-integrated features should plug into the -daemon capability path and `banger doctor` checks first, with the remaining -shell helpers treated as manual workflows rather than architecture drivers. -- Stopping a VM preserves its overlay and work disk. - -## Rebuilding The Repo Default Rootfs -`config/packages.apt` controls the base apt packages baked into rebuilt images, -including guest tools such as `ss` used by `banger vm ports`. - -To rebuild the source-checkout default image in `./build/runtime/rootfs-docker.ext4`: ```bash -make rootfs +./build/bin/banger daemon status ``` -That rebuild also regenerates `./build/runtime/rootfs-docker.work-seed.ext4`, which -the daemon uses to speed up future `vm create` calls, and bakes in the default -host-reachable `opencode` server service. +Disable it with: -If your runtime bundle does not include `./build/runtime/rootfs.ext4`, pass an -explicit base image instead: -```bash -./scripts/make-rootfs.sh --base-rootfs /path/to/base-rootfs.ext4 +```toml +web_listen_addr = "" ``` -If the package manifest changed and you want a fresh source-checkout image: +## Guest Services + +Provisioned images include: + +- `banger-vsock-agent` +- guest networking bootstrap +- `mise` +- `opencode` +- a default guest `opencode` service on `0.0.0.0:4096` + +From the host: + ```bash -rm -f ./build/runtime/rootfs-docker.ext4 ./build/runtime/rootfs-docker.ext4.packages.sha256 -make rootfs +./build/bin/banger vm ports testbox +opencode attach http://:4096 ``` -`make rootfs` expects a bootstrapped runtime bundle. If `./build/runtime/rootfs.ext4` -is not available, pass an explicit `--base-rootfs` to `./scripts/make-rootfs.sh`. -Existing VMs keep using their current image and disks; rebuilds only affect VMs -created from the rebuilt image afterward. Restarting an existing VM is not -enough to pick up guest provisioning changes such as the default `opencode` -server service. +## Manual Helpers + +The shell helpers are now explicit manual workflows under `./build/manual`. + +Rebuild a Debian-style manual rootfs: + +```bash +make rootfs ARGS='--base-rootfs /abs/path/rootfs.ext4 --kernel /abs/path/vmlinux --initrd /abs/path/initrd.img --modules /abs/path/modules' +``` + +The output lands in: + +- `./build/manual/rootfs-docker.ext4` +- `./build/manual/rootfs-docker.work-seed.ext4` + +## Experimental Void Flow + +Stage a Void kernel: -## Experimental Void Rootfs -There is also a separate, opt-in builder for an experimental Void Linux guest -path: ```bash make void-kernel +``` + +Build the experimental Void rootfs: + +```bash make rootfs-void ``` -That writes: -- `./build/runtime/void-kernel/` when `make void-kernel` is used -- `./build/runtime/rootfs-void.ext4` -- `./build/runtime/rootfs-void.work-seed.ext4` +Register it: -This path is intentionally local-only and does not change the default Debian -image flow. `make void-kernel` stages an actual Void `linux6.12` kernel package -under `./build/runtime/void-kernel/`, including the raw `vmlinuz`, extracted -Firecracker `vmlinux`, a matching `initramfs`, the matching config, and the -matching modules tree. The initramfs is generated locally with `dracut` -against the downloaded Void sysroot so the kernel, initrd, and modules stay -aligned. `make rootfs-void` then prefers that staged modules tree when it exists; -otherwise it falls back to the runtime bundle modules. The rootfs builder -itself still builds a lean `x86_64-glibc` Void userspace with: -- `bash` installed for interactive/admin use -- pinned `mise` installed at `/usr/local/bin/mise`, activated for `root` bash shells -- `opencode` installed through `mise`, with `/usr/local/bin/opencode` available by default -- a guest network bootstrap that configures the VM NIC from the kernel `ip=` boot arg -- a host-reachable `opencode serve` runit service enabled on guest TCP port `4096` -- `docker` plus `docker-compose` installed from Void packages -- the `docker` runit service enabled, with Docker netfilter/forwarding kernel prep -- `openssh` enabled under runit -- the bundled `banger-vsock-agent` health agent enabled under runit -- `root` normalized to `/bin/bash` while keeping `/bin/sh` as the distro's system shell -- a generated `/root` work-seed for fast creates - -It still keeps some Debian-oriented extras out for now: -- no tmux plugin defaults - -The builder fetches official static XBPS tools and packages from the Void -mirror during the build. The kernel fetcher and rootfs builder currently -support only `x86_64`. - -The package set comes from [`config/packages.void`](/home/thales/projects/personal/banger/config/packages.void). -You can override the mirror, size, output path, or kernel package directly: ```bash -./scripts/make-void-kernel.sh --kernel-package linux6.12 -./scripts/make-rootfs-void.sh --mirror https://repo-default.voidlinux.org --size 2G -``` - -The fastest local iteration loop does not require changing your default image -config at all: -```bash -make void-kernel -make rootfs-void make void-register -./build/bin/banger vm create --image void-exp --name void-dev -./build/bin/banger vm ssh void-dev ``` -Rebuild the staged Void kernel or Void rootfs, then recreate existing -`void-exp` VMs after changing the package set, guest provisioning, or staged -kernel artifacts; restart alone will not update the image contents, kernel, or -`/root` work-seed. +That flow uses: -There is also a smoke path for the experimental image: -```bash -make verify-void -``` +- `./build/manual/void-kernel/` +- `./build/manual/rootfs-void.ext4` +- `./build/manual/rootfs-void.work-seed.ext4` -`make void-register` uses the unmanaged image registration path to create or -update a `void-exp` image record in place, so repeated rebuilds do not require -editing `~/.config/banger/config.toml`. It expects a complete staged Void -kernel set under `./build/runtime/void-kernel/` and points the experimental image at -the staged Void `vmlinux`, `initramfs`, and matching modules tree. +## Notes -There is also a one-step helper target: -```bash -make void-vm VOID_VM_NAME=void-a -``` - -If you really want the Void image to become your default for `vm create` -without `--image`, use the checked-in override template at -[`examples/void-exp.config.toml`](/home/thales/projects/personal/banger/examples/void-exp.config.toml) -and merge its four settings into `~/.config/banger/config.toml`. - -`banger image build` remains Debian-only in this pass. Do not point -`default_base_rootfs` at the Void artifact yet. - -## Registering Unmanaged Images -You can also register any local rootfs as an unmanaged image record without -changing global defaults: -```bash -banger image register --name local-test --rootfs /abs/path/rootfs.ext4 -``` - -Optional paths let you point at an existing work seed, kernel, initrd, modules, -and package manifest: -```bash -banger image register \ - --name void-exp \ - --rootfs ./build/runtime/rootfs-void.ext4 \ - --work-seed ./build/runtime/rootfs-void.work-seed.ext4 \ - --kernel ./build/runtime/void-kernel/boot/vmlinux-6.12.77_1 \ - --initrd ./build/runtime/void-kernel/boot/initramfs-6.12.77_1.img \ - --modules ./build/runtime/void-kernel/lib/modules/6.12.77_1 \ - --packages ./config/packages.void -``` - -If an unmanaged image with the same name already exists, `image register` -updates it in place so future `vm create --image ` calls pick up the new -artifacts immediately. - -## Maintaining The Runtime Bundle -The checked-in [`config/runtime-bundle.toml`](/home/thales/projects/personal/banger/config/runtime-bundle.toml) -is a template. Keep `bundle_metadata` accurate there, but use a separate local -manifest copy when you need concrete `url` and `sha256` values for bootstrap -testing or publication. - -Package a local `./build/runtime/` tree into an archive: -```bash -make runtime-package -``` - -That writes `build/dist/banger-runtime.tar.gz` and prints its SHA256 so you can update -a local manifest copy before testing bootstrap changes or publishing the -archive elsewhere. - -## Benchmarking Create Time -Benchmark the current host's `vm create` wall time plus first-SSH readiness: -```bash -make bench-create -``` - -Pass options through `ARGS`, for example: -```bash -make bench-create ARGS="--runs 3 --image docker-dev" -``` - -The benchmark prints JSON with: -- `create_ms`: wall time for `banger vm create`, including full readiness - gating for the guest vsock agent and default `opencode` service -- `ssh_ready_ms`: wall time from create start until `banger vm ssh -- true` - succeeds - -## Remaining Shell Helpers -The runtime VM lifecycle is managed through `banger`. The remaining shell scripts are not the primary user interface: -- `scripts/customize.sh`: manual reference flow for rootfs customization; `banger image build` is now Go-native, but the script still reads - assets from `BANGER_RUNTIME_DIR` and stores transient state under - `BANGER_STATE_DIR`/XDG state -- `scripts/make-rootfs.sh`: convenience wrapper for rebuilding `./build/runtime/rootfs-docker.ext4` -- `scripts/interactive.sh`: manual one-off rootfs customization over SSH -- `scripts/lib/packages.sh`: shell helper library -- `scripts/verify.sh`: smoke test for the Go workflow (`./scripts/verify.sh --nat` adds NAT coverage) +- Firecracker is resolved from `PATH` by default. +- Managed image delete removes the daemon-owned artifact dir. +- The companion vsock helper is internal to the install/build layout, not a user-configured runtime path. diff --git a/cmd/runtimebundle/main.go b/cmd/runtimebundle/main.go deleted file mode 100644 index 28da9ea..0000000 --- a/cmd/runtimebundle/main.go +++ /dev/null @@ -1,78 +0,0 @@ -package main - -import ( - "context" - "flag" - "fmt" - "os" - - "banger/internal/runtimebundle" -) - -const ( - defaultManifestPath = "config/runtime-bundle.toml" - defaultRuntimeDir = "build/runtime" - defaultArchivePath = "build/dist/banger-runtime.tar.gz" -) - -func main() { - if len(os.Args) < 2 { - usage() - os.Exit(2) - } - switch os.Args[1] { - case "fetch": - if err := fetch(os.Args[2:]); err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } - case "package": - if err := pkg(os.Args[2:]); err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } - default: - usage() - os.Exit(2) - } -} - -func fetch(args []string) error { - fs := flag.NewFlagSet("fetch", flag.ContinueOnError) - fs.SetOutput(os.Stderr) - manifestPath := fs.String("manifest", defaultManifestPath, "path to the runtime bundle manifest") - outDir := fs.String("out", defaultRuntimeDir, "destination runtime directory") - if err := fs.Parse(args); err != nil { - return err - } - manifest, err := runtimebundle.LoadManifest(*manifestPath) - if err != nil { - return err - } - return runtimebundle.Bootstrap(context.Background(), manifest, *manifestPath, *outDir) -} - -func pkg(args []string) error { - fs := flag.NewFlagSet("package", flag.ContinueOnError) - fs.SetOutput(os.Stderr) - manifestPath := fs.String("manifest", defaultManifestPath, "path to the runtime bundle manifest") - runtimeDir := fs.String("runtime-dir", defaultRuntimeDir, "runtime directory to package") - outArchive := fs.String("out", defaultArchivePath, "output archive path") - if err := fs.Parse(args); err != nil { - return err - } - manifest, err := runtimebundle.LoadManifest(*manifestPath) - if err != nil { - return err - } - sum, err := runtimebundle.Package(*runtimeDir, *outArchive, manifest) - if err != nil { - return err - } - fmt.Println(sum) - return nil -} - -func usage() { - fmt.Fprintln(os.Stderr, "usage: runtimebundle [flags]") -} diff --git a/config/packages.apt b/config/packages.apt deleted file mode 100644 index 54a5159..0000000 --- a/config/packages.apt +++ /dev/null @@ -1,10 +0,0 @@ -make -git -less -tree -ca-certificates -curl -wget -iproute2 -vim -tmux diff --git a/config/packages.void b/config/packages.void deleted file mode 100644 index 2b3b99e..0000000 --- a/config/packages.void +++ /dev/null @@ -1,27 +0,0 @@ -base-minimal -base-devel -bash -openssh -ca-certificates -curl -docker -docker-compose -fd -fzf -git -iputils -jq -kmod -iproute2 -less -lsof -make -procps-ng -psmisc -ripgrep -strace -tmux -vim -unzip -zip -zstd diff --git a/config/runtime-bundle.toml b/config/runtime-bundle.toml deleted file mode 100644 index 460cb53..0000000 --- a/config/runtime-bundle.toml +++ /dev/null @@ -1,33 +0,0 @@ -# Template manifest for local or published runtime bundle archives. -# Keep this checked-in file empty by default; use a local manifest copy with -# concrete `url` and `sha256` values when bootstrapping `./build/runtime/`. -version = "v0" -url = "" -sha256 = "" -bundle_root = "runtime" -required_paths = [ - "firecracker", - "customize.sh", - "packages.sh", - "namegen", - "banger-vsock-agent", - "packages.apt", - "id_ed25519", - "rootfs-docker.ext4", - "wtf/root/boot/vmlinux-6.8.0-94-generic", - "wtf/root/boot/initrd.img-6.8.0-94-generic", - "wtf/root/lib/modules/6.8.0-94-generic", -] - -[bundle_metadata] -firecracker_bin = "firecracker" -ssh_key_path = "id_ed25519" -namegen_path = "namegen" -customize_script = "customize.sh" -vsock_agent_path = "banger-vsock-agent" -default_packages_file = "packages.apt" -default_rootfs = "rootfs-docker.ext4" -default_work_seed = "rootfs-docker.work-seed.ext4" -default_kernel = "wtf/root/boot/vmlinux-6.8.0-94-generic" -default_initrd = "wtf/root/boot/initrd.img-6.8.0-94-generic" -default_modules_dir = "wtf/root/lib/modules/6.8.0-94-generic" diff --git a/examples/void-exp.config.toml b/examples/void-exp.config.toml index 192f433..1266ada 100644 --- a/examples/void-exp.config.toml +++ b/examples/void-exp.config.toml @@ -1,14 +1,9 @@ # Experimental Void Linux guest profile for local testing. # -# Copy the values you want into ~/.config/banger/config.toml and replace -# /abs/path/to/banger with your checkout path. Do not set default_base_rootfs -# to the Void image yet; banger image build still assumes the Debian flow. -# If you run `make void-kernel`, also merge the commented kernel/initrd/modules lines. +# Register or promote a complete `void-exp` image first, then point the daemon +# at it by name. Firecracker is resolved from PATH by default; set +# `firecracker_bin` only if you need an override. -runtime_dir = "/abs/path/to/banger/build/runtime" default_image_name = "void-exp" -default_rootfs = "/abs/path/to/banger/build/runtime/rootfs-void.ext4" -default_work_seed = "/abs/path/to/banger/build/runtime/rootfs-void.work-seed.ext4" -# default_kernel = "/abs/path/to/banger/build/runtime/void-kernel/boot/vmlinux-6.12.77_1" -# default_initrd = "/abs/path/to/banger/build/runtime/void-kernel/boot/initramfs-6.12.77_1.img" -# default_modules_dir = "/abs/path/to/banger/build/runtime/void-kernel/lib/modules/6.12.77_1" +# firecracker_bin = "/usr/bin/firecracker" +# ssh_key_path = "/abs/path/to/private/key" diff --git a/internal/api/types.go b/internal/api/types.go index ca44542..fcd6961 100644 --- a/internal/api/types.go +++ b/internal/api/types.go @@ -149,7 +149,7 @@ type VMPortsResult struct { type ImageBuildParams struct { Name string `json:"name,omitempty"` - BaseRootfs string `json:"base_rootfs,omitempty"` + FromImage string `json:"from_image,omitempty"` Size string `json:"size,omitempty"` KernelPath string `json:"kernel_path,omitempty"` InitrdPath string `json:"initrd_path,omitempty"` @@ -164,7 +164,6 @@ type ImageRegisterParams struct { KernelPath string `json:"kernel_path,omitempty"` InitrdPath string `json:"initrd_path,omitempty"` ModulesDir string `json:"modules_dir,omitempty"` - PackagesPath string `json:"packages_path,omitempty"` Docker bool `json:"docker,omitempty"` } diff --git a/internal/cli/banger.go b/internal/cli/banger.go index 89154e0..761172b 100644 --- a/internal/cli/banger.go +++ b/internal/cli/banger.go @@ -20,6 +20,7 @@ import ( "banger/internal/config" "banger/internal/daemon" "banger/internal/hostnat" + "banger/internal/imagepreset" "banger/internal/model" "banger/internal/paths" "banger/internal/rpc" @@ -101,7 +102,104 @@ func newInternalCommand() *cobra.Command { Hidden: true, RunE: helpNoArgs, } - cmd.AddCommand(newInternalNATCommand(), newInternalWorkSeedCommand()) + cmd.AddCommand( + newInternalNATCommand(), + newInternalWorkSeedCommand(), + newInternalSSHKeyPathCommand(), + newInternalFirecrackerPathCommand(), + newInternalVSockAgentPathCommand(), + newInternalPackagesCommand(), + ) + return cmd +} + +func newInternalSSHKeyPathCommand() *cobra.Command { + return &cobra.Command{ + Use: "ssh-key-path", + Hidden: true, + Args: noArgsUsage("usage: banger internal ssh-key-path"), + RunE: func(cmd *cobra.Command, args []string) error { + layout, err := paths.Resolve() + if err != nil { + return err + } + cfg, err := config.Load(layout) + if err != nil { + return err + } + _, err = fmt.Fprintln(cmd.OutOrStdout(), cfg.SSHKeyPath) + return err + }, + } +} + +func newInternalFirecrackerPathCommand() *cobra.Command { + return &cobra.Command{ + Use: "firecracker-path", + Hidden: true, + Args: noArgsUsage("usage: banger internal firecracker-path"), + RunE: func(cmd *cobra.Command, args []string) error { + layout, err := paths.Resolve() + if err != nil { + return err + } + cfg, err := config.Load(layout) + if err != nil { + return err + } + if strings.TrimSpace(cfg.FirecrackerBin) == "" { + return errors.New("firecracker binary not configured; install firecracker or set firecracker_bin") + } + _, err = fmt.Fprintln(cmd.OutOrStdout(), cfg.FirecrackerBin) + return err + }, + } +} + +func newInternalVSockAgentPathCommand() *cobra.Command { + return &cobra.Command{ + Use: "vsock-agent-path", + Hidden: true, + Args: noArgsUsage("usage: banger internal vsock-agent-path"), + RunE: func(cmd *cobra.Command, args []string) error { + path, err := paths.CompanionBinaryPath("banger-vsock-agent") + if err != nil { + return err + } + _, err = fmt.Fprintln(cmd.OutOrStdout(), path) + return err + }, + } +} + +func newInternalPackagesCommand() *cobra.Command { + var docker bool + cmd := &cobra.Command{ + Use: "packages ", + Hidden: true, + Args: exactArgsUsage(1, "usage: banger internal packages [--docker]"), + RunE: func(cmd *cobra.Command, args []string) error { + var packages []string + switch strings.TrimSpace(args[0]) { + case "debian": + packages = imagepreset.DebianBasePackages() + if docker { + packages = append(packages, "docker.io") + } + case "void": + packages = imagepreset.VoidBasePackages() + default: + return fmt.Errorf("unknown package preset %q", args[0]) + } + for _, pkg := range packages { + if _, err := fmt.Fprintln(cmd.OutOrStdout(), pkg); err != nil { + return err + } + } + return nil + }, + } + cmd.Flags().BoolVar(&docker, "docker", false, "include docker-specific additions") return cmd } @@ -630,7 +728,7 @@ func newImageBuildCommand() *cobra.Command { }, } cmd.Flags().StringVar(¶ms.Name, "name", "", "image name") - cmd.Flags().StringVar(¶ms.BaseRootfs, "base-rootfs", "", "base rootfs path") + cmd.Flags().StringVar(¶ms.FromImage, "from-image", "", "registered base image id or name") cmd.Flags().StringVar(¶ms.Size, "size", "", "output image size") cmd.Flags().StringVar(¶ms.KernelPath, "kernel", "", "kernel path") cmd.Flags().StringVar(¶ms.InitrdPath, "initrd", "", "initrd path") @@ -644,7 +742,7 @@ func newImageRegisterCommand() *cobra.Command { cmd := &cobra.Command{ Use: "register", Short: "Register or update an unmanaged image", - Args: noArgsUsage("usage: banger image register --name --rootfs [--work-seed ] [--kernel ] [--initrd ] [--modules ] [--packages ]"), + Args: noArgsUsage("usage: banger image register --name --rootfs [--work-seed ] --kernel [--initrd ] [--modules ]"), RunE: func(cmd *cobra.Command, args []string) error { if err := absolutizeImageRegisterPaths(¶ms); err != nil { return err @@ -669,7 +767,6 @@ func newImageRegisterCommand() *cobra.Command { cmd.Flags().StringVar(¶ms.KernelPath, "kernel", "", "kernel path") cmd.Flags().StringVar(¶ms.InitrdPath, "initrd", "", "initrd path") cmd.Flags().StringVar(¶ms.ModulesDir, "modules", "", "modules dir") - cmd.Flags().StringVar(¶ms.PackagesPath, "packages", "", "packages manifest path") cmd.Flags().BoolVar(¶ms.Docker, "docker", false, "mark image as docker-prepared") return cmd } @@ -1158,13 +1255,13 @@ func validateSSHPrereqs(cfg model.DaemonConfig) error { checks := system.NewPreflight() checks.RequireCommand("ssh", "install openssh-client") if strings.TrimSpace(cfg.SSHKeyPath) != "" { - checks.RequireFile(cfg.SSHKeyPath, "runtime ssh private key", `refresh the runtime bundle`) + checks.RequireFile(cfg.SSHKeyPath, "ssh private key", `set "ssh_key_path" or let banger create its default key`) } return checks.Err("ssh preflight failed") } func absolutizeImageBuildPaths(params *api.ImageBuildParams) error { - return absolutizePaths(¶ms.BaseRootfs, ¶ms.KernelPath, ¶ms.InitrdPath, ¶ms.ModulesDir) + return absolutizePaths(¶ms.KernelPath, ¶ms.InitrdPath, ¶ms.ModulesDir) } func absolutizeImageRegisterPaths(params *api.ImageRegisterParams) error { @@ -1174,7 +1271,6 @@ func absolutizeImageRegisterPaths(params *api.ImageRegisterParams) error { ¶ms.KernelPath, ¶ms.InitrdPath, ¶ms.ModulesDir, - ¶ms.PackagesPath, ) } diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index b0b0f63..f82b76d 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -163,7 +163,7 @@ func TestImageRegisterFlagsExist(t *testing.T) { if err != nil { t.Fatalf("find register: %v", err) } - for _, flagName := range []string{"name", "rootfs", "work-seed", "kernel", "initrd", "modules", "packages", "docker"} { + for _, flagName := range []string{"name", "rootfs", "work-seed", "kernel", "initrd", "modules", "docker"} { if register.Flags().Lookup(flagName) == nil { t.Fatalf("missing flag %q", flagName) } @@ -427,7 +427,6 @@ func TestAbsolutizeImageRegisterPaths(t *testing.T) { KernelPath: filepath.Join(".", "runtime", "vmlinux"), InitrdPath: filepath.Join(".", "runtime", "initrd.img"), ModulesDir: filepath.Join(".", "runtime", "modules"), - PackagesPath: filepath.Join(".", "config", "packages.void"), } wd, err := os.Getwd() @@ -450,7 +449,6 @@ func TestAbsolutizeImageRegisterPaths(t *testing.T) { params.KernelPath, params.InitrdPath, params.ModulesDir, - params.PackagesPath, } { if !filepath.IsAbs(value) { t.Fatalf("path %q is not absolute", value) @@ -828,7 +826,7 @@ func TestAbsolutizeImageBuildPaths(t *testing.T) { }) params := api.ImageBuildParams{ - BaseRootfs: "images/base.ext4", + FromImage: "base-image", KernelPath: "/kernel", InitrdPath: "boot/initrd.img", ModulesDir: "modules", @@ -838,7 +836,7 @@ func TestAbsolutizeImageBuildPaths(t *testing.T) { } want := api.ImageBuildParams{ - BaseRootfs: filepath.Join(dir, "images/base.ext4"), + FromImage: "base-image", KernelPath: "/kernel", InitrdPath: filepath.Join(dir, "boot/initrd.img"), ModulesDir: filepath.Join(dir, "modules"), diff --git a/internal/config/config.go b/internal/config/config.go index ebdca41..bfaf926 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -1,38 +1,29 @@ package config import ( - "errors" + "crypto/ed25519" + "crypto/rand" + "crypto/x509" + "encoding/pem" "os" "path/filepath" "strings" "time" toml "github.com/pelletier/go-toml" + "golang.org/x/crypto/ssh" "banger/internal/model" "banger/internal/paths" - "banger/internal/runtimebundle" + "banger/internal/system" ) type fileConfig struct { - RuntimeDir string `toml:"runtime_dir"` - RepoRoot string `toml:"repo_root"` LogLevel string `toml:"log_level"` WebListenAddr *string `toml:"web_listen_addr"` FirecrackerBin string `toml:"firecracker_bin"` SSHKeyPath string `toml:"ssh_key_path"` - NamegenPath string `toml:"namegen_path"` - CustomizeScript string `toml:"customize_script"` - VSockAgent string `toml:"vsock_agent_path"` - VSockPingHelper string `toml:"vsock_ping_helper_path"` - DefaultWorkSeed string `toml:"default_work_seed"` DefaultImageName string `toml:"default_image_name"` - DefaultRootfs string `toml:"default_rootfs"` - DefaultBaseRootfs string `toml:"default_base_rootfs"` - DefaultKernel string `toml:"default_kernel"` - DefaultInitrd string `toml:"default_initrd"` - DefaultModulesDir string `toml:"default_modules_dir"` - DefaultPackages string `toml:"default_packages_file"` AutoStopStaleAfter string `toml:"auto_stop_stale_after"` StatsPollInterval string `toml:"stats_poll_interval"` MetricsPoll string `toml:"metrics_poll_interval"` @@ -58,202 +49,130 @@ func Load(layout paths.Layout) (model.DaemonConfig, error) { DefaultImageName: "default", } - path := filepath.Join(layout.ConfigDir, "config.toml") - info, err := os.Stat(path) var file fileConfig - if err != nil { - if !os.IsNotExist(err) { - return cfg, err - } - } else if !info.IsDir() { - data, err := os.ReadFile(path) + configPath := filepath.Join(layout.ConfigDir, "config.toml") + if info, err := os.Stat(configPath); err == nil && !info.IsDir() { + data, err := os.ReadFile(configPath) if err != nil { return cfg, err } if err := toml.Unmarshal(data, &file); err != nil { return cfg, err } - } - - cfg.RuntimeDir = paths.ResolveRuntimeDir(file.RuntimeDir, file.RepoRoot) - if err := applyRuntimeDefaults(&cfg); err != nil { + } else if err != nil && !os.IsNotExist(err) { return cfg, err } - if file.FirecrackerBin != "" { - cfg.FirecrackerBin = file.FirecrackerBin - } - if file.LogLevel != "" { - cfg.LogLevel = file.LogLevel + if value := strings.TrimSpace(file.LogLevel); value != "" { + cfg.LogLevel = value } if file.WebListenAddr != nil { cfg.WebListenAddr = strings.TrimSpace(*file.WebListenAddr) } - if file.NamegenPath != "" { - cfg.NamegenPath = file.NamegenPath + if value := strings.TrimSpace(file.FirecrackerBin); value != "" { + cfg.FirecrackerBin = value + } else if path, err := system.LookupExecutable("firecracker"); err == nil { + cfg.FirecrackerBin = path } - if file.CustomizeScript != "" { - cfg.CustomizeScript = file.CustomizeScript + if value := strings.TrimSpace(file.DefaultImageName); value != "" { + cfg.DefaultImageName = value } - if file.VSockAgent != "" { - cfg.VSockAgentPath = file.VSockAgent - } else if file.VSockPingHelper != "" { - cfg.VSockAgentPath = file.VSockPingHelper + if value := strings.TrimSpace(file.BridgeName); value != "" { + cfg.BridgeName = value } - if file.DefaultWorkSeed != "" { - cfg.DefaultWorkSeed = file.DefaultWorkSeed + if value := strings.TrimSpace(file.BridgeIP); value != "" { + cfg.BridgeIP = value } - if file.DefaultImageName != "" { - cfg.DefaultImageName = file.DefaultImageName - } - if file.DefaultRootfs != "" { - cfg.DefaultRootfs = file.DefaultRootfs - } - if file.DefaultBaseRootfs != "" { - cfg.DefaultBaseRootfs = file.DefaultBaseRootfs - } - if file.DefaultKernel != "" { - cfg.DefaultKernel = file.DefaultKernel - } - if file.DefaultInitrd != "" { - cfg.DefaultInitrd = file.DefaultInitrd - } - if file.DefaultModulesDir != "" { - cfg.DefaultModulesDir = file.DefaultModulesDir - } - if file.DefaultPackages != "" { - cfg.DefaultPackagesFile = file.DefaultPackages - } - if file.BridgeName != "" { - cfg.BridgeName = file.BridgeName - } - if file.BridgeIP != "" { - cfg.BridgeIP = file.BridgeIP - } - if file.CIDR != "" { - cfg.CIDR = file.CIDR + if value := strings.TrimSpace(file.CIDR); value != "" { + cfg.CIDR = value } if file.TapPoolSize > 0 { cfg.TapPoolSize = file.TapPoolSize } - if file.DefaultDNS != "" { - cfg.DefaultDNS = file.DefaultDNS + if value := strings.TrimSpace(file.DefaultDNS); value != "" { + cfg.DefaultDNS = value } - if file.AutoStopStaleAfter != "" { - duration, err := time.ParseDuration(file.AutoStopStaleAfter) + if value := strings.TrimSpace(file.AutoStopStaleAfter); value != "" { + duration, err := time.ParseDuration(value) if err != nil { return cfg, err } cfg.AutoStopStaleAfter = duration } - if file.StatsPollInterval != "" { - duration, err := time.ParseDuration(file.StatsPollInterval) + if value := strings.TrimSpace(file.StatsPollInterval); value != "" { + duration, err := time.ParseDuration(value) if err != nil { return cfg, err } cfg.StatsPollInterval = duration } - if file.MetricsPoll != "" { - duration, err := time.ParseDuration(file.MetricsPoll) + if value := strings.TrimSpace(file.MetricsPoll); value != "" { + duration, err := time.ParseDuration(value) if err != nil { return cfg, err } cfg.MetricsPollInterval = duration } - if value := os.Getenv("BANGER_LOG_LEVEL"); value != "" { + if value := strings.TrimSpace(os.Getenv("BANGER_LOG_LEVEL")); value != "" { cfg.LogLevel = value } + + sshKeyPath, err := resolveSSHKeyPath(layout, file.SSHKeyPath) + if err != nil { + return cfg, err + } + cfg.SSHKeyPath = sshKeyPath return cfg, nil } -func applyRuntimeDefaults(cfg *model.DaemonConfig) error { - if cfg.RuntimeDir == "" { - return nil +func resolveSSHKeyPath(layout paths.Layout, configured string) (string, error) { + configured = strings.TrimSpace(configured) + if configured != "" { + return configured, nil } - meta, err := runtimebundle.LoadBundleMetadata(cfg.RuntimeDir) - switch { - case err == nil: - applyBundleMetadataDefaults(cfg, cfg.RuntimeDir, meta) - case errors.Is(err, os.ErrNotExist): - applyLegacyRuntimeDefaults(cfg) - default: + return ensureDefaultSSHKey(filepath.Join(layout.ConfigDir, "ssh", "id_ed25519")) +} + +func ensureDefaultSSHKey(path string) (string, error) { + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + return "", err + } + if _, err := os.Stat(path); err == nil { + if err := ensurePublicKeyFile(path); err != nil { + return "", err + } + return path, nil + } else if !os.IsNotExist(err) { + return "", err + } + + _, privateKey, err := ed25519.GenerateKey(rand.Reader) + if err != nil { + return "", err + } + pkcs8, err := x509.MarshalPKCS8PrivateKey(privateKey) + if err != nil { + return "", err + } + privatePEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: pkcs8}) + if err := os.WriteFile(path, privatePEM, 0o600); err != nil { + return "", err + } + if err := ensurePublicKeyFile(path); err != nil { + return "", err + } + return path, nil +} + +func ensurePublicKeyFile(privateKeyPath string) error { + data, err := os.ReadFile(privateKeyPath) + if err != nil { return err } - if cfg.DefaultRootfs == "" { - cfg.DefaultRootfs = firstExistingRuntimePath( - filepath.Join(cfg.RuntimeDir, "rootfs-docker.ext4"), - filepath.Join(cfg.RuntimeDir, "rootfs.ext4"), - ) + signer, err := ssh.ParsePrivateKey(data) + if err != nil { + return err } - if cfg.DefaultBaseRootfs == "" { - cfg.DefaultBaseRootfs = firstExistingRuntimePath( - filepath.Join(cfg.RuntimeDir, "rootfs.ext4"), - cfg.DefaultRootfs, - ) - } - if cfg.DefaultWorkSeed == "" && cfg.DefaultRootfs != "" { - cfg.DefaultWorkSeed = firstExistingRuntimePath(associatedWorkSeedPath(cfg.DefaultRootfs)) - } - return nil -} - -func applyBundleMetadataDefaults(cfg *model.DaemonConfig, runtimeDir string, meta runtimebundle.BundleMetadata) { - cfg.FirecrackerBin = defaultRuntimePath(cfg.FirecrackerBin, runtimeDir, meta.FirecrackerBin) - cfg.SSHKeyPath = defaultRuntimePath(cfg.SSHKeyPath, runtimeDir, meta.SSHKeyPath) - cfg.NamegenPath = defaultRuntimePath(cfg.NamegenPath, runtimeDir, meta.NamegenPath) - cfg.CustomizeScript = defaultRuntimePath(cfg.CustomizeScript, runtimeDir, meta.CustomizeScript) - cfg.VSockAgentPath = defaultRuntimePath(cfg.VSockAgentPath, runtimeDir, meta.VSockAgentPath) - cfg.DefaultWorkSeed = defaultRuntimePath(cfg.DefaultWorkSeed, runtimeDir, meta.DefaultWorkSeed) - cfg.DefaultKernel = defaultRuntimePath(cfg.DefaultKernel, runtimeDir, meta.DefaultKernel) - cfg.DefaultInitrd = defaultRuntimePath(cfg.DefaultInitrd, runtimeDir, meta.DefaultInitrd) - cfg.DefaultModulesDir = defaultRuntimePath(cfg.DefaultModulesDir, runtimeDir, meta.DefaultModulesDir) - cfg.DefaultPackagesFile = defaultRuntimePath(cfg.DefaultPackagesFile, runtimeDir, meta.DefaultPackages) - cfg.DefaultRootfs = defaultRuntimePath(cfg.DefaultRootfs, runtimeDir, meta.DefaultRootfs) - cfg.DefaultBaseRootfs = defaultRuntimePath(cfg.DefaultBaseRootfs, runtimeDir, meta.DefaultBaseRootfs) -} - -func applyLegacyRuntimeDefaults(cfg *model.DaemonConfig) { - cfg.FirecrackerBin = defaultRuntimePath(cfg.FirecrackerBin, cfg.RuntimeDir, "firecracker") - cfg.SSHKeyPath = defaultRuntimePath(cfg.SSHKeyPath, cfg.RuntimeDir, "id_ed25519") - cfg.NamegenPath = defaultRuntimePath(cfg.NamegenPath, cfg.RuntimeDir, "namegen") - cfg.CustomizeScript = defaultRuntimePath(cfg.CustomizeScript, cfg.RuntimeDir, "customize.sh") - cfg.VSockAgentPath = firstExistingRuntimePath( - defaultRuntimePath(cfg.VSockAgentPath, cfg.RuntimeDir, "banger-vsock-agent"), - filepath.Join(cfg.RuntimeDir, "banger-vsock-pingd"), - ) - cfg.DefaultWorkSeed = defaultRuntimePath(cfg.DefaultWorkSeed, cfg.RuntimeDir, "rootfs-docker.work-seed.ext4") - cfg.DefaultKernel = defaultRuntimePath(cfg.DefaultKernel, cfg.RuntimeDir, "wtf/root/boot/vmlinux-6.8.0-94-generic") - cfg.DefaultInitrd = defaultRuntimePath(cfg.DefaultInitrd, cfg.RuntimeDir, "wtf/root/boot/initrd.img-6.8.0-94-generic") - cfg.DefaultModulesDir = defaultRuntimePath(cfg.DefaultModulesDir, cfg.RuntimeDir, "wtf/root/lib/modules/6.8.0-94-generic") - cfg.DefaultPackagesFile = defaultRuntimePath(cfg.DefaultPackagesFile, cfg.RuntimeDir, "packages.apt") -} - -func defaultRuntimePath(current, runtimeDir, relative string) string { - if current != "" || relative == "" { - return current - } - return filepath.Join(runtimeDir, relative) -} - -func firstExistingRuntimePath(paths ...string) string { - for _, candidate := range paths { - if candidate == "" { - continue - } - if _, err := os.Stat(candidate); err == nil { - return candidate - } - } - return "" -} - -func associatedWorkSeedPath(rootfsPath string) string { - rootfsPath = strings.TrimSpace(rootfsPath) - if rootfsPath == "" { - return "" - } - if strings.HasSuffix(rootfsPath, ".ext4") { - return strings.TrimSuffix(rootfsPath, ".ext4") + ".work-seed.ext4" - } - return rootfsPath + ".work-seed" + publicKey := ssh.MarshalAuthorizedKey(signer.PublicKey()) + return os.WriteFile(privateKeyPath+".pub", publicKey, 0o644) } diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 665ab9b..1934c6a 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -1,154 +1,70 @@ package config import ( - "encoding/json" "os" "path/filepath" "testing" + "time" "banger/internal/paths" - "banger/internal/runtimebundle" ) -func TestLoadDerivesArtifactPathsFromRuntimeDir(t *testing.T) { - runtimeDir := t.TempDir() - meta := runtimebundle.BundleMetadata{ - FirecrackerBin: "bin/firecracker", - SSHKeyPath: "keys/id_ed25519", - NamegenPath: "bin/namegen", - CustomizeScript: "scripts/customize.sh", - VSockAgentPath: "bin/banger-vsock-agent", - DefaultPackages: "config/packages.apt", - DefaultRootfs: "images/rootfs-docker.ext4", - DefaultWorkSeed: "images/rootfs-docker.work-seed.ext4", - DefaultKernel: "kernels/vmlinux", - DefaultInitrd: "kernels/initrd.img", - DefaultModulesDir: "modules/current", - } - for _, rel := range []string{ - meta.FirecrackerBin, - meta.SSHKeyPath, - meta.NamegenPath, - meta.CustomizeScript, - meta.VSockAgentPath, - meta.DefaultPackages, - meta.DefaultRootfs, - meta.DefaultWorkSeed, - meta.DefaultKernel, - meta.DefaultInitrd, - filepath.Join(meta.DefaultModulesDir, "modules.dep"), - } { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) - } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) - } - } - data, err := json.Marshal(meta) - if err != nil { - t.Fatalf("Marshal: %v", err) - } - if err := os.WriteFile(filepath.Join(runtimeDir, runtimebundle.BundleMetadataFile), data, 0o644); err != nil { - t.Fatalf("write bundle metadata: %v", err) +func TestLoadDefaultsResolveFirecrackerAndGenerateSSHKey(t *testing.T) { + configDir := t.TempDir() + binDir := t.TempDir() + firecrackerPath := filepath.Join(binDir, "firecracker") + if err := os.WriteFile(firecrackerPath, []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { + t.Fatalf("write firecracker: %v", err) } + t.Setenv("PATH", binDir) - t.Setenv("BANGER_RUNTIME_DIR", runtimeDir) - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) + cfg, err := Load(paths.Layout{ConfigDir: configDir}) if err != nil { t.Fatalf("Load: %v", err) } - if cfg.RuntimeDir != runtimeDir { - t.Fatalf("RuntimeDir = %q, want %q", cfg.RuntimeDir, runtimeDir) + if cfg.FirecrackerBin != firecrackerPath { + t.Fatalf("FirecrackerBin = %q, want %q", cfg.FirecrackerBin, firecrackerPath) } - if cfg.FirecrackerBin != filepath.Join(runtimeDir, meta.FirecrackerBin) { - t.Fatalf("FirecrackerBin = %q", cfg.FirecrackerBin) + wantKey := filepath.Join(configDir, "ssh", "id_ed25519") + if cfg.SSHKeyPath != wantKey { + t.Fatalf("SSHKeyPath = %q, want %q", cfg.SSHKeyPath, wantKey) } - if cfg.SSHKeyPath != filepath.Join(runtimeDir, meta.SSHKeyPath) { - t.Fatalf("SSHKeyPath = %q", cfg.SSHKeyPath) + for _, path := range []string{wantKey, wantKey + ".pub"} { + if _, err := os.Stat(path); err != nil { + t.Fatalf("stat %s: %v", path, err) + } } - if cfg.NamegenPath != filepath.Join(runtimeDir, meta.NamegenPath) { - t.Fatalf("NamegenPath = %q", cfg.NamegenPath) + if cfg.DefaultImageName != "default" { + t.Fatalf("DefaultImageName = %q, want default", cfg.DefaultImageName) } - if cfg.CustomizeScript != filepath.Join(runtimeDir, meta.CustomizeScript) { - t.Fatalf("CustomizeScript = %q", cfg.CustomizeScript) - } - if cfg.VSockAgentPath != filepath.Join(runtimeDir, meta.VSockAgentPath) { - t.Fatalf("VSockAgentPath = %q", cfg.VSockAgentPath) - } - if cfg.DefaultRootfs != filepath.Join(runtimeDir, meta.DefaultRootfs) { - t.Fatalf("DefaultRootfs = %q", cfg.DefaultRootfs) - } - if cfg.DefaultWorkSeed != filepath.Join(runtimeDir, meta.DefaultWorkSeed) { - t.Fatalf("DefaultWorkSeed = %q", cfg.DefaultWorkSeed) - } - if cfg.DefaultBaseRootfs != filepath.Join(runtimeDir, meta.DefaultRootfs) { - t.Fatalf("DefaultBaseRootfs = %q", cfg.DefaultBaseRootfs) - } - if cfg.DefaultKernel != filepath.Join(runtimeDir, meta.DefaultKernel) { - t.Fatalf("DefaultKernel = %q", cfg.DefaultKernel) - } - if cfg.DefaultInitrd != filepath.Join(runtimeDir, meta.DefaultInitrd) { - t.Fatalf("DefaultInitrd = %q", cfg.DefaultInitrd) - } - if cfg.DefaultModulesDir != filepath.Join(runtimeDir, meta.DefaultModulesDir) { - t.Fatalf("DefaultModulesDir = %q", cfg.DefaultModulesDir) - } - if cfg.DefaultPackagesFile != filepath.Join(runtimeDir, meta.DefaultPackages) { - t.Fatalf("DefaultPackagesFile = %q", cfg.DefaultPackagesFile) + if cfg.WebListenAddr != "127.0.0.1:7777" { + t.Fatalf("WebListenAddr = %q", cfg.WebListenAddr) } } -func TestLoadFallsBackToLegacyRuntimeLayoutWithoutBundleMetadata(t *testing.T) { - runtimeDir := t.TempDir() - for _, rel := range []string{ - "firecracker", - "id_ed25519", - "namegen", - "customize.sh", - "banger-vsock-agent", - "packages.apt", - "rootfs-docker.ext4", - "rootfs-docker.work-seed.ext4", - "wtf/root/boot/vmlinux-6.8.0-94-generic", - "wtf/root/boot/initrd.img-6.8.0-94-generic", - "wtf/root/lib/modules/6.8.0-94-generic/modules.dep", - } { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) - } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) - } +func TestLoadAppliesConfigOverrides(t *testing.T) { + configDir := t.TempDir() + data := []byte(` +log_level = "debug" +web_listen_addr = "" +firecracker_bin = "/opt/firecracker" +ssh_key_path = "/tmp/custom-key" +default_image_name = "void-exp" +auto_stop_stale_after = "1h" +stats_poll_interval = "15s" +metrics_poll_interval = "30s" +bridge_name = "br-test" +bridge_ip = "10.0.0.1" +cidr = "25" +tap_pool_size = 8 +default_dns = "9.9.9.9" +`) + if err := os.WriteFile(filepath.Join(configDir, "config.toml"), data, 0o644); err != nil { + t.Fatalf("write config.toml: %v", err) } - t.Setenv("BANGER_RUNTIME_DIR", runtimeDir) - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) - if err != nil { - t.Fatalf("Load: %v", err) - } - - if cfg.FirecrackerBin != filepath.Join(runtimeDir, "firecracker") { - t.Fatalf("FirecrackerBin = %q", cfg.FirecrackerBin) - } - if cfg.VSockAgentPath != filepath.Join(runtimeDir, "banger-vsock-agent") { - t.Fatalf("VSockAgentPath = %q", cfg.VSockAgentPath) - } - if cfg.DefaultWorkSeed != filepath.Join(runtimeDir, "rootfs-docker.work-seed.ext4") { - t.Fatalf("DefaultWorkSeed = %q", cfg.DefaultWorkSeed) - } - if cfg.DefaultKernel != filepath.Join(runtimeDir, "wtf/root/boot/vmlinux-6.8.0-94-generic") { - t.Fatalf("DefaultKernel = %q", cfg.DefaultKernel) - } -} - -func TestLoadAppliesLogLevelEnvOverride(t *testing.T) { - t.Setenv("BANGER_LOG_LEVEL", "debug") - - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) + cfg, err := Load(paths.Layout{ConfigDir: configDir}) if err != nil { t.Fatalf("Load: %v", err) } @@ -156,158 +72,46 @@ func TestLoadAppliesLogLevelEnvOverride(t *testing.T) { if cfg.LogLevel != "debug" { t.Fatalf("LogLevel = %q", cfg.LogLevel) } -} - -func TestLoadDefaultsLogLevelToInfo(t *testing.T) { - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) - if err != nil { - t.Fatalf("Load: %v", err) - } - if cfg.LogLevel != "info" { - t.Fatalf("LogLevel = %q, want info", cfg.LogLevel) - } -} - -func TestLoadIgnoresConfigSSHKeyOverrideForGuestAccess(t *testing.T) { - runtimeDir := t.TempDir() - meta := runtimebundle.BundleMetadata{ - FirecrackerBin: "bin/firecracker", - SSHKeyPath: "keys/id_ed25519", - NamegenPath: "bin/namegen", - CustomizeScript: "scripts/customize.sh", - VSockAgentPath: "bin/banger-vsock-agent", - DefaultPackages: "config/packages.apt", - DefaultRootfs: "images/rootfs.ext4", - DefaultWorkSeed: "images/rootfs.work-seed.ext4", - DefaultKernel: "kernels/vmlinux", - DefaultModulesDir: "modules/current", - } - for _, rel := range []string{ - meta.FirecrackerBin, - meta.SSHKeyPath, - meta.NamegenPath, - meta.CustomizeScript, - meta.VSockAgentPath, - meta.DefaultPackages, - meta.DefaultRootfs, - meta.DefaultWorkSeed, - meta.DefaultKernel, - filepath.Join(meta.DefaultModulesDir, "modules.dep"), - } { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) - } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) - } - } - data, err := json.Marshal(meta) - if err != nil { - t.Fatalf("Marshal: %v", err) - } - if err := os.WriteFile(filepath.Join(runtimeDir, runtimebundle.BundleMetadataFile), data, 0o644); err != nil { - t.Fatalf("write bundle metadata: %v", err) - } - - configDir := t.TempDir() - if err := os.WriteFile(filepath.Join(configDir, "config.toml"), []byte("ssh_key_path = \"/tmp/override-key\"\n"), 0o644); err != nil { - t.Fatalf("write config.toml: %v", err) - } - - t.Setenv("BANGER_RUNTIME_DIR", runtimeDir) - cfg, err := Load(paths.Layout{ConfigDir: configDir}) - if err != nil { - t.Fatalf("Load: %v", err) - } - - want := filepath.Join(runtimeDir, meta.SSHKeyPath) - if cfg.SSHKeyPath != want { - t.Fatalf("SSHKeyPath = %q, want runtime key %q", cfg.SSHKeyPath, want) - } -} - -func TestLoadAcceptsLegacyBundleVsockPingHelperPath(t *testing.T) { - runtimeDir := t.TempDir() - meta := runtimebundle.BundleMetadata{ - FirecrackerBin: "bin/firecracker", - SSHKeyPath: "keys/id_ed25519", - NamegenPath: "bin/namegen", - CustomizeScript: "scripts/customize.sh", - VSockPingHelperPath: "bin/banger-vsock-pingd", - DefaultPackages: "config/packages.apt", - DefaultRootfs: "images/rootfs.ext4", - DefaultKernel: "kernels/vmlinux", - } - for _, rel := range []string{ - meta.FirecrackerBin, - meta.SSHKeyPath, - meta.NamegenPath, - meta.CustomizeScript, - meta.VSockPingHelperPath, - meta.DefaultPackages, - meta.DefaultRootfs, - meta.DefaultKernel, - } { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) - } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) - } - } - data, err := json.Marshal(meta) - if err != nil { - t.Fatalf("Marshal: %v", err) - } - if err := os.WriteFile(filepath.Join(runtimeDir, runtimebundle.BundleMetadataFile), data, 0o644); err != nil { - t.Fatalf("write bundle metadata: %v", err) - } - - t.Setenv("BANGER_RUNTIME_DIR", runtimeDir) - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) - if err != nil { - t.Fatalf("Load: %v", err) - } - if cfg.VSockAgentPath != filepath.Join(runtimeDir, meta.VSockPingHelperPath) { - t.Fatalf("VSockAgentPath = %q", cfg.VSockAgentPath) - } -} - -func TestLoadAcceptsLegacyConfigVsockPingHelperPath(t *testing.T) { - configDir := t.TempDir() - if err := os.WriteFile(filepath.Join(configDir, "config.toml"), []byte("vsock_ping_helper_path = \"/tmp/legacy-agent\"\n"), 0o644); err != nil { - t.Fatalf("write config.toml: %v", err) - } - - cfg, err := Load(paths.Layout{ConfigDir: configDir}) - if err != nil { - t.Fatalf("Load: %v", err) - } - if cfg.VSockAgentPath != "/tmp/legacy-agent" { - t.Fatalf("VSockAgentPath = %q", cfg.VSockAgentPath) - } -} - -func TestLoadWebListenAddrDefaultsAndAllowsDisable(t *testing.T) { - cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) - if err != nil { - t.Fatalf("Load default config: %v", err) - } - if cfg.WebListenAddr != "127.0.0.1:7777" { - t.Fatalf("WebListenAddr = %q, want default 127.0.0.1:7777", cfg.WebListenAddr) - } - - configDir := t.TempDir() - if err := os.WriteFile(filepath.Join(configDir, "config.toml"), []byte("web_listen_addr = \"\"\n"), 0o644); err != nil { - t.Fatalf("write config.toml: %v", err) - } - cfg, err = Load(paths.Layout{ConfigDir: configDir}) - if err != nil { - t.Fatalf("Load disabled config: %v", err) - } if cfg.WebListenAddr != "" { - t.Fatalf("WebListenAddr = %q, want disabled empty string", cfg.WebListenAddr) + t.Fatalf("WebListenAddr = %q, want empty", cfg.WebListenAddr) + } + if cfg.FirecrackerBin != "/opt/firecracker" { + t.Fatalf("FirecrackerBin = %q", cfg.FirecrackerBin) + } + if cfg.SSHKeyPath != "/tmp/custom-key" { + t.Fatalf("SSHKeyPath = %q", cfg.SSHKeyPath) + } + if cfg.DefaultImageName != "void-exp" { + t.Fatalf("DefaultImageName = %q", cfg.DefaultImageName) + } + if cfg.AutoStopStaleAfter != time.Hour { + t.Fatalf("AutoStopStaleAfter = %s", cfg.AutoStopStaleAfter) + } + if cfg.StatsPollInterval != 15*time.Second { + t.Fatalf("StatsPollInterval = %s", cfg.StatsPollInterval) + } + if cfg.MetricsPollInterval != 30*time.Second { + t.Fatalf("MetricsPollInterval = %s", cfg.MetricsPollInterval) + } + if cfg.BridgeName != "br-test" || cfg.BridgeIP != "10.0.0.1" || cfg.CIDR != "25" { + t.Fatalf("bridge config = %+v", cfg) + } + if cfg.TapPoolSize != 8 { + t.Fatalf("TapPoolSize = %d", cfg.TapPoolSize) + } + if cfg.DefaultDNS != "9.9.9.9" { + t.Fatalf("DefaultDNS = %q", cfg.DefaultDNS) + } +} + +func TestLoadAppliesLogLevelEnvOverride(t *testing.T) { + t.Setenv("BANGER_LOG_LEVEL", "warn") + + cfg, err := Load(paths.Layout{ConfigDir: t.TempDir()}) + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.LogLevel != "warn" { + t.Fatalf("LogLevel = %q, want warn", cfg.LogLevel) } } diff --git a/internal/daemon/capabilities.go b/internal/daemon/capabilities.go index 0b866bc..d2ec524 100644 --- a/internal/daemon/capabilities.go +++ b/internal/daemon/capabilities.go @@ -208,11 +208,13 @@ func (workDiskCapability) PrepareHost(ctx context.Context, d *Daemon, vm *model. } func (workDiskCapability) AddDoctorChecks(_ context.Context, d *Daemon, report *system.Report) { - if strings.TrimSpace(d.config.DefaultWorkSeed) != "" && exists(d.config.DefaultWorkSeed) { - checks := system.NewPreflight() - checks.RequireFile(d.config.DefaultWorkSeed, "default work seed image", `rebuild the default runtime rootfs to regenerate the /root seed`) - report.AddPreflight("feature /root work disk", checks, "seeded /root work disk artifact available") - return + if d.store != nil && strings.TrimSpace(d.config.DefaultImageName) != "" { + if image, err := d.store.GetImageByName(context.Background(), d.config.DefaultImageName); err == nil && strings.TrimSpace(image.WorkSeedPath) != "" && exists(image.WorkSeedPath) { + checks := system.NewPreflight() + checks.RequireFile(image.WorkSeedPath, "default image work-seed", `rebuild the default image to regenerate the /root seed`) + report.AddPreflight("feature /root work disk", checks, "seeded /root work disk artifact available") + return + } } checks := system.NewPreflight() for _, command := range []string{"mkfs.ext4", "mount", "umount", "cp"} { diff --git a/internal/daemon/daemon.go b/internal/daemon/daemon.go index 1042caf..e142fd3 100644 --- a/internal/daemon/daemon.go +++ b/internal/daemon/daemon.go @@ -11,7 +11,6 @@ import ( "net" "net/http" "os" - "path/filepath" "strings" "sync" "time" @@ -85,7 +84,7 @@ func Open(ctx context.Context) (d *Daemon, err error) { closing: make(chan struct{}), pid: os.Getpid(), } - d.logger.Info("daemon opened", "socket", layout.SocketPath, "state_dir", layout.StateDir, "runtime_dir", cfg.RuntimeDir, "log_level", cfg.LogLevel) + d.logger.Info("daemon opened", "socket", layout.SocketPath, "state_dir", layout.StateDir, "log_level", cfg.LogLevel) if err = d.startVMDNS(vmdns.DefaultListenAddr); err != nil { d.logger.Error("daemon open failed", "stage", "start_vm_dns", "error", err.Error()) return nil, err @@ -95,10 +94,6 @@ func Open(ctx context.Context) (d *Daemon, err error) { _ = d.stopVMDNS() } }() - if err = d.ensureDefaultImage(ctx); err != nil { - d.logger.Error("daemon open failed", "stage", "ensure_default_image", "error", err.Error()) - return nil, err - } if err = d.reconcile(ctx); err != nil { d.logger.Error("daemon open failed", "stage", "reconcile", "error", err.Error()) return nil, err @@ -499,95 +494,8 @@ func (d *Daemon) stopVMDNS() error { } func (d *Daemon) ensureDefaultImage(ctx context.Context) error { - if d.config.DefaultImageName == "" { - return nil - } - desired, ok := d.desiredDefaultImage() - if !ok { - if d.logger != nil { - d.logger.Debug("default image skipped", "image_name", d.config.DefaultImageName, "rootfs_path", d.config.DefaultRootfs, "kernel_path", d.config.DefaultKernel) - } - return nil - } - - image, err := d.store.GetImageByName(ctx, d.config.DefaultImageName) - switch { - case err == nil: - if image.Managed { - if d.logger != nil { - d.logger.Debug("managed default image left untouched", append(imageLogAttrs(image), "managed", image.Managed)...) - } - return nil - } - if defaultImageMatches(image, desired) { - if d.logger != nil { - d.logger.Debug("default image already current", imageLogAttrs(image)...) - } - return nil - } - updated := desired - updated.ID = image.ID - updated.CreatedAt = image.CreatedAt - updated.UpdatedAt = model.Now() - if err := d.store.UpsertImage(ctx, updated); err != nil { - return err - } - if d.logger != nil { - d.logger.Info("default image reconciled", append(imageLogAttrs(updated), "previous_rootfs_path", image.RootfsPath, "previous_work_seed_path", image.WorkSeedPath, "previous_kernel_path", image.KernelPath)...) - } - return nil - case errors.Is(err, sql.ErrNoRows): - id, err := model.NewID() - if err != nil { - return err - } - now := model.Now() - desired.ID = id - desired.CreatedAt = now - desired.UpdatedAt = now - if err := d.store.UpsertImage(ctx, desired); err != nil { - return err - } - if d.logger != nil { - d.logger.Info("default image registered", append(imageLogAttrs(desired), "managed", desired.Managed)...) - } - return nil - default: - return err - } -} - -func (d *Daemon) desiredDefaultImage() (model.Image, bool) { - rootfs := d.config.DefaultRootfs - kernel := d.config.DefaultKernel - if !exists(rootfs) || !exists(kernel) { - return model.Image{}, false - } - return model.Image{ - Name: d.config.DefaultImageName, - Managed: false, - ArtifactDir: "", - RootfsPath: rootfs, - WorkSeedPath: d.config.DefaultWorkSeed, - KernelPath: kernel, - InitrdPath: d.config.DefaultInitrd, - ModulesDir: d.config.DefaultModulesDir, - PackagesPath: d.config.DefaultPackagesFile, - Docker: strings.Contains(filepath.Base(rootfs), "docker"), - }, true -} - -func defaultImageMatches(current, desired model.Image) bool { - return current.Name == desired.Name && - current.Managed == desired.Managed && - current.ArtifactDir == desired.ArtifactDir && - current.RootfsPath == desired.RootfsPath && - current.WorkSeedPath == desired.WorkSeedPath && - current.KernelPath == desired.KernelPath && - current.InitrdPath == desired.InitrdPath && - current.ModulesDir == desired.ModulesDir && - current.PackagesPath == desired.PackagesPath && - current.Docker == desired.Docker + _ = ctx + return nil } func (d *Daemon) reconcile(ctx context.Context) error { diff --git a/internal/daemon/daemon_test.go b/internal/daemon/daemon_test.go index 82ffa7f..dc43c59 100644 --- a/internal/daemon/daemon_test.go +++ b/internal/daemon/daemon_test.go @@ -1,722 +1,106 @@ package daemon import ( - "bufio" - "bytes" "context" - "encoding/json" - "net" "os" "path/filepath" "strings" "testing" - "time" "banger/internal/api" "banger/internal/model" "banger/internal/paths" - "banger/internal/rpc" - "banger/internal/store" + "banger/internal/system" ) -func TestEnsureDefaultImageUsesConfiguredDefaultRootfs(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, _, _, _ := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - +func TestBuildImageRequiresFromImage(t *testing.T) { d := &Daemon{ - config: model.DaemonConfig{ - DefaultImageName: "default", - DefaultRootfs: rootfs, - DefaultKernel: kernel, - }, - store: db, + layout: paths.Layout{ImagesDir: t.TempDir(), StateDir: t.TempDir()}, + store: openDaemonStore(t), + runner: system.NewRunner(), } - if err := d.ensureDefaultImage(context.Background()); err != nil { - t.Fatalf("ensureDefaultImage: %v", err) - } - - image, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if image.RootfsPath != rootfs { - t.Fatalf("RootfsPath = %q, want %q", image.RootfsPath, rootfs) - } - if image.KernelPath != kernel { - t.Fatalf("KernelPath = %q, want %q", image.KernelPath, kernel) - } - if image.Managed { - t.Fatal("default image should be unmanaged") + _, err := d.BuildImage(context.Background(), api.ImageBuildParams{Name: "missing-base"}) + if err == nil || !strings.Contains(err.Error(), "from-image is required") { + t.Fatalf("BuildImage() error = %v", err) } } -func TestEnsureDefaultImageLeavesCurrentUnmanagedDefaultUntouched(t *testing.T) { +func TestRegisterImageRequiresKernel(t *testing.T) { + rootfs := filepath.Join(t.TempDir(), "rootfs.ext4") + if err := os.WriteFile(rootfs, []byte("rootfs"), 0o644); err != nil { + t.Fatalf("write rootfs: %v", err) + } + d := &Daemon{store: openDaemonStore(t)} + + _, err := d.RegisterImage(context.Background(), api.ImageRegisterParams{ + Name: "missing-kernel", + RootfsPath: rootfs, + }) + if err == nil || !strings.Contains(err.Error(), "kernel path is required") { + t.Fatalf("RegisterImage() error = %v", err) + } +} + +func TestPromoteImageCopiesBootArtifactsIntoArtifactDir(t *testing.T) { dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, packages := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) + rootfs := filepath.Join(dir, "rootfs.ext4") + kernel := filepath.Join(dir, "vmlinux") + initrd := filepath.Join(dir, "initrd.img") + modulesDir := filepath.Join(dir, "modules") + if err := os.MkdirAll(modulesDir, 0o755); err != nil { + t.Fatalf("mkdir modules: %v", err) + } + for path, data := range map[string]string{ + rootfs: "rootfs", + kernel: "kernel", + initrd: "initrd", + filepath.Join(modulesDir, "depmod"): "modules", + } { + if err := os.WriteFile(path, []byte(data), 0o644); err != nil { + t.Fatalf("write %s: %v", path, err) + } + } + + db := openDaemonStore(t) image := model.Image{ - ID: "default-id", - Name: "default", - Managed: false, - RootfsPath: rootfs, - KernelPath: kernel, - InitrdPath: initrd, - ModulesDir: modulesDir, - PackagesPath: packages, - Docker: true, - CreatedAt: now, - UpdatedAt: now, + ID: "img-promote", + Name: "void-exp", + Managed: false, + RootfsPath: rootfs, + KernelPath: kernel, + InitrdPath: initrd, + ModulesDir: modulesDir, + CreatedAt: model.Now(), + UpdatedAt: model.Now(), } if err := db.UpsertImage(context.Background(), image); err != nil { t.Fatalf("UpsertImage: %v", err) } - d := &Daemon{ - config: model.DaemonConfig{ - DefaultImageName: "default", - DefaultRootfs: rootfs, - DefaultKernel: kernel, - DefaultInitrd: initrd, - DefaultModulesDir: modulesDir, - DefaultPackagesFile: packages, - }, - store: db, - } - - if err := d.ensureDefaultImage(context.Background()); err != nil { - t.Fatalf("ensureDefaultImage: %v", err) - } - - got, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if got.ID != image.ID { - t.Fatalf("ID = %q, want %q", got.ID, image.ID) - } - if !got.UpdatedAt.Equal(image.UpdatedAt) { - t.Fatalf("UpdatedAt = %s, want unchanged %s", got.UpdatedAt, image.UpdatedAt) - } -} - -func TestEnsureDefaultImageReconcilesStaleUnmanagedDefaultInPlace(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, packages := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) - stale := model.Image{ - ID: "default-id", - Name: "default", - Managed: false, - RootfsPath: "/home/thales/projects/personal/banger/build/runtime/rootfs-docker.ext4", - KernelPath: "/home/thales/projects/personal/banger/build/runtime/wtf/root/boot/vmlinux-6.8.0-94-generic", - InitrdPath: "/home/thales/projects/personal/banger/build/runtime/wtf/root/boot/initrd.img-6.8.0-94-generic", - ModulesDir: "/home/thales/projects/personal/banger/build/runtime/wtf/root/lib/modules/6.8.0-94-generic", - PackagesPath: "/home/thales/projects/personal/banger/build/runtime/packages.apt", - Docker: true, - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), stale); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - vm := testVM("uses-default", stale.ID, "172.16.0.25") - if err := db.UpsertVM(context.Background(), vm); err != nil { - t.Fatalf("UpsertVM: %v", err) + imagesDir := filepath.Join(dir, "images") + if err := os.MkdirAll(imagesDir, 0o755); err != nil { + t.Fatalf("mkdir images dir: %v", err) } d := &Daemon{ - config: model.DaemonConfig{ - DefaultImageName: "default", - DefaultRootfs: rootfs, - DefaultKernel: kernel, - DefaultInitrd: initrd, - DefaultModulesDir: modulesDir, - DefaultPackagesFile: packages, - }, - store: db, - } - - if err := d.ensureDefaultImage(context.Background()); err != nil { - t.Fatalf("ensureDefaultImage: %v", err) - } - - got, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if got.ID != stale.ID { - t.Fatalf("ID = %q, want preserved %q", got.ID, stale.ID) - } - if !got.CreatedAt.Equal(stale.CreatedAt) { - t.Fatalf("CreatedAt = %s, want preserved %s", got.CreatedAt, stale.CreatedAt) - } - if got.RootfsPath != rootfs || got.KernelPath != kernel || got.InitrdPath != initrd || got.ModulesDir != modulesDir || got.PackagesPath != packages { - t.Fatalf("stale default not reconciled: %+v", got) - } - if !got.UpdatedAt.After(stale.UpdatedAt) { - t.Fatalf("UpdatedAt = %s, want newer than %s", got.UpdatedAt, stale.UpdatedAt) - } - gotVM, err := db.GetVMByID(context.Background(), vm.ID) - if err != nil { - t.Fatalf("GetVMByID: %v", err) - } - if gotVM.ImageID != stale.ID { - t.Fatalf("VM image ID = %q, want preserved %q", gotVM.ImageID, stale.ID) - } -} - -func TestEnsureDefaultImageLeavesManagedDefaultUntouched(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, _, _, _ := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) - managed := model.Image{ - ID: "managed-default", - Name: "default", - Managed: true, - RootfsPath: "/managed/rootfs.ext4", - KernelPath: "/managed/vmlinux", - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), managed); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - - d := &Daemon{ - config: model.DaemonConfig{ - DefaultImageName: "default", - DefaultRootfs: rootfs, - DefaultKernel: kernel, - }, - store: db, - } - - if err := d.ensureDefaultImage(context.Background()); err != nil { - t.Fatalf("ensureDefaultImage: %v", err) - } - - got, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if got.RootfsPath != managed.RootfsPath || got.KernelPath != managed.KernelPath { - t.Fatalf("managed default was rewritten: %+v", got) - } -} - -func TestEnsureDefaultImageSkipsRewriteWhenCurrentArtifactsMissing(t *testing.T) { - dir := t.TempDir() - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) - stale := model.Image{ - ID: "default-id", - Name: "default", - Managed: false, - RootfsPath: "/old/rootfs.ext4", - KernelPath: "/old/vmlinux", - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), stale); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - - d := &Daemon{ - config: model.DaemonConfig{ - DefaultImageName: "default", - DefaultRootfs: filepath.Join(dir, "missing-rootfs.ext4"), - DefaultKernel: filepath.Join(dir, "missing-vmlinux"), - }, - store: db, - } - - if err := d.ensureDefaultImage(context.Background()); err != nil { - t.Fatalf("ensureDefaultImage: %v", err) - } - - got, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if got.RootfsPath != stale.RootfsPath || got.KernelPath != stale.KernelPath { - t.Fatalf("default image should have stayed stale when no current artifacts exist: %+v", got) - } -} - -func TestRegisterImageCreatesUnmanagedImage(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, _ := writeDefaultImageArtifacts(t, dir) - workSeed := filepath.Join(dir, "rootfs-void.work-seed.ext4") - packages := filepath.Join(dir, "packages.void") - if err := os.WriteFile(workSeed, []byte("seed"), 0o644); err != nil { - t.Fatalf("WriteFile(workSeed): %v", err) - } - if err := os.WriteFile(packages, []byte("base-minimal\nopenssh\n"), 0o644); err != nil { - t.Fatalf("WriteFile(packages): %v", err) - } - db := openDefaultImageStore(t, dir) - d := &Daemon{ - config: model.DaemonConfig{ - DefaultKernel: kernel, - DefaultInitrd: initrd, - DefaultModulesDir: modulesDir, - }, - store: db, - } - - image, err := d.RegisterImage(context.Background(), api.ImageRegisterParams{ - Name: "void-exp", - RootfsPath: rootfs, - WorkSeedPath: workSeed, - PackagesPath: packages, - }) - if err != nil { - t.Fatalf("RegisterImage: %v", err) - } - if image.Managed { - t.Fatal("registered image should be unmanaged") - } - if image.Name != "void-exp" || image.RootfsPath != rootfs || image.WorkSeedPath != workSeed || image.KernelPath != kernel { - t.Fatalf("registered image = %+v", image) - } -} - -func TestRegisterImageUpdatesExistingUnmanagedImageInPlace(t *testing.T) { - dir := t.TempDir() - _, kernel, initrd, modulesDir, _ := writeDefaultImageArtifacts(t, dir) - newRootfs := filepath.Join(dir, "rootfs-void-next.ext4") - newWorkSeed := filepath.Join(dir, "rootfs-void-next.work-seed.ext4") - packages := filepath.Join(dir, "packages.void") - for _, path := range []string{newRootfs, newWorkSeed} { - if err := os.WriteFile(path, []byte("next"), 0o644); err != nil { - t.Fatalf("WriteFile(%s): %v", path, err) - } - } - if err := os.WriteFile(packages, []byte("base-minimal\n"), 0o644); err != nil { - t.Fatalf("WriteFile(packages): %v", err) - } - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) - existing := model.Image{ - ID: "void-image-id", - Name: "void-exp", - Managed: false, - RootfsPath: filepath.Join(dir, "old-rootfs.ext4"), - KernelPath: kernel, - InitrdPath: initrd, - ModulesDir: modulesDir, - PackagesPath: packages, - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), existing); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - d := &Daemon{ - config: model.DaemonConfig{ - DefaultKernel: kernel, - DefaultInitrd: initrd, - DefaultModulesDir: modulesDir, - }, - store: db, - } - - image, err := d.RegisterImage(context.Background(), api.ImageRegisterParams{ - Name: "void-exp", - RootfsPath: newRootfs, - WorkSeedPath: newWorkSeed, - PackagesPath: packages, - }) - if err != nil { - t.Fatalf("RegisterImage: %v", err) - } - if image.ID != existing.ID || !image.CreatedAt.Equal(existing.CreatedAt) { - t.Fatalf("updated image identity changed: %+v", image) - } - if image.RootfsPath != newRootfs || image.WorkSeedPath != newWorkSeed { - t.Fatalf("updated image paths not applied: %+v", image) - } -} - -func TestRegisterImageRejectsManagedOverwrite(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, _, _, _ := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 16, 12, 0, 0, 0, time.UTC) - if err := db.UpsertImage(context.Background(), model.Image{ - ID: "managed-id", - Name: "void-exp", - Managed: true, - RootfsPath: rootfs, - KernelPath: kernel, - CreatedAt: now, - UpdatedAt: now, - }); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - d := &Daemon{config: model.DaemonConfig{DefaultKernel: kernel}, store: db} - - _, err := d.RegisterImage(context.Background(), api.ImageRegisterParams{ - Name: "void-exp", - RootfsPath: rootfs, - }) - if err == nil || !strings.Contains(err.Error(), "cannot be updated via register") { - t.Fatalf("RegisterImage(managed) error = %v", err) - } -} - -func TestPromoteImageCopiesArtifactsAndPreservesIdentity(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, packages := writeDefaultImageArtifacts(t, dir) - workSeed := filepath.Join(dir, "rootfs-docker.work-seed.ext4") - workSeedContent := []byte("seed-data") - if err := os.WriteFile(workSeed, workSeedContent, 0o644); err != nil { - t.Fatalf("WriteFile(workSeed): %v", err) - } - - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 20, 12, 0, 0, 0, time.UTC) - existing := model.Image{ - ID: "promote-image-id", - Name: "default", - Managed: false, - RootfsPath: rootfs, - WorkSeedPath: workSeed, - KernelPath: kernel, - InitrdPath: initrd, - ModulesDir: modulesDir, - PackagesPath: packages, - Docker: true, - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), existing); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - vm := testVM("uses-default", existing.ID, "172.16.0.44") - if err := db.UpsertVM(context.Background(), vm); err != nil { - t.Fatalf("UpsertVM: %v", err) - } - - d := &Daemon{ - layout: modelPathsLayoutForTest(dir), + layout: paths.Layout{ImagesDir: imagesDir}, store: db, + runner: system.NewRunner(), } - - image, err := d.PromoteImage(context.Background(), "default") + got, err := d.PromoteImage(context.Background(), image.Name) if err != nil { t.Fatalf("PromoteImage: %v", err) } - if !image.Managed { + if !got.Managed { t.Fatal("promoted image should be managed") } - if image.ID != existing.ID || image.Name != existing.Name { - t.Fatalf("promoted image identity changed: %+v", image) - } - if !image.CreatedAt.Equal(existing.CreatedAt) { - t.Fatalf("CreatedAt = %s, want preserved %s", image.CreatedAt, existing.CreatedAt) - } - if !image.UpdatedAt.After(existing.UpdatedAt) { - t.Fatalf("UpdatedAt = %s, want newer than %s", image.UpdatedAt, existing.UpdatedAt) - } - wantArtifactDir := filepath.Join(d.layout.ImagesDir, existing.ID) - if image.ArtifactDir != wantArtifactDir { - t.Fatalf("ArtifactDir = %q, want %q", image.ArtifactDir, wantArtifactDir) - } - if image.RootfsPath != filepath.Join(wantArtifactDir, "rootfs.ext4") { - t.Fatalf("RootfsPath = %q, want managed copy", image.RootfsPath) - } - if image.WorkSeedPath != filepath.Join(wantArtifactDir, "work-seed.ext4") { - t.Fatalf("WorkSeedPath = %q, want managed copy", image.WorkSeedPath) - } - if image.KernelPath != kernel || image.InitrdPath != initrd || image.ModulesDir != modulesDir || image.PackagesPath != packages { - t.Fatalf("boot support paths changed unexpectedly: %+v", image) - } - - rootfsContent, err := os.ReadFile(rootfs) - if err != nil { - t.Fatalf("ReadFile(rootfs): %v", err) - } - managedRootfsContent, err := os.ReadFile(image.RootfsPath) - if err != nil { - t.Fatalf("ReadFile(managed rootfs): %v", err) - } - if !bytes.Equal(managedRootfsContent, rootfsContent) { - t.Fatal("managed rootfs copy content mismatch") - } - managedWorkSeedContent, err := os.ReadFile(image.WorkSeedPath) - if err != nil { - t.Fatalf("ReadFile(managed work seed): %v", err) - } - if !bytes.Equal(managedWorkSeedContent, workSeedContent) { - t.Fatal("managed work seed copy content mismatch") - } - - got, err := db.GetImageByName(context.Background(), "default") - if err != nil { - t.Fatalf("GetImageByName: %v", err) - } - if got.RootfsPath != image.RootfsPath || !got.Managed || got.ArtifactDir != image.ArtifactDir { - t.Fatalf("stored promoted image = %+v, want %+v", got, image) - } - gotVM, err := db.GetVMByID(context.Background(), vm.ID) - if err != nil { - t.Fatalf("GetVMByID: %v", err) - } - if gotVM.ImageID != existing.ID { - t.Fatalf("VM image ID = %q, want preserved %q", gotVM.ImageID, existing.ID) - } -} - -func TestPromoteImageRejectsManagedImage(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, packages := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 20, 12, 0, 0, 0, time.UTC) - if err := db.UpsertImage(context.Background(), model.Image{ - ID: "managed-id", - Name: "default", - Managed: true, - ArtifactDir: filepath.Join(dir, "images", "managed-id"), - RootfsPath: rootfs, - KernelPath: kernel, - InitrdPath: initrd, - ModulesDir: modulesDir, - PackagesPath: packages, - CreatedAt: now, - UpdatedAt: now, - }); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - d := &Daemon{ - layout: modelPathsLayoutForTest(dir), - store: db, - } - - _, err := d.PromoteImage(context.Background(), "default") - if err == nil || !strings.Contains(err.Error(), "already managed") { - t.Fatalf("PromoteImage(managed) error = %v", err) - } -} - -func TestPromoteImageSkipsMissingWorkSeed(t *testing.T) { - dir := t.TempDir() - rootfs, kernel, initrd, modulesDir, packages := writeDefaultImageArtifacts(t, dir) - db := openDefaultImageStore(t, dir) - now := time.Date(2026, time.March, 20, 12, 0, 0, 0, time.UTC) - existing := model.Image{ - ID: "promote-missing-seed", - Name: "default", - Managed: false, - RootfsPath: rootfs, - WorkSeedPath: filepath.Join(dir, "missing.work-seed.ext4"), - KernelPath: kernel, - InitrdPath: initrd, - ModulesDir: modulesDir, - PackagesPath: packages, - CreatedAt: now, - UpdatedAt: now, - } - if err := db.UpsertImage(context.Background(), existing); err != nil { - t.Fatalf("UpsertImage: %v", err) - } - d := &Daemon{ - layout: modelPathsLayoutForTest(dir), - store: db, - } - - image, err := d.PromoteImage(context.Background(), "default") - if err != nil { - t.Fatalf("PromoteImage: %v", err) - } - if image.WorkSeedPath != "" { - t.Fatalf("WorkSeedPath = %q, want empty for missing source work seed", image.WorkSeedPath) - } - if _, err := os.Stat(filepath.Join(image.ArtifactDir, "work-seed.ext4")); !os.IsNotExist(err) { - t.Fatalf("managed work-seed should not exist, stat error = %v", err) - } -} - -func openDefaultImageStore(t *testing.T, dir string) *store.Store { - t.Helper() - db, err := store.Open(filepath.Join(dir, "state.db")) - if err != nil { - t.Fatalf("open store: %v", err) - } - t.Cleanup(func() { - _ = db.Close() - }) - return db -} - -func writeDefaultImageArtifacts(t *testing.T, dir string) (rootfs, kernel, initrd, modulesDir, packages string) { - t.Helper() - rootfs = filepath.Join(dir, "rootfs-docker.ext4") - kernel = filepath.Join(dir, "vmlinux") - initrd = filepath.Join(dir, "initrd.img") - modulesDir = filepath.Join(dir, "modules") - packages = filepath.Join(dir, "packages.apt") - files := []string{ - rootfs, - kernel, - initrd, - packages, - filepath.Join(modulesDir, "modules.dep"), - } - for _, path := range files { - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) + for _, path := range []string{got.RootfsPath, got.KernelPath, got.InitrdPath, got.ModulesDir} { + if !strings.HasPrefix(path, got.ArtifactDir) { + t.Fatalf("artifact path %q does not live under %q", path, got.ArtifactDir) } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) + if _, err := os.Stat(path); err != nil { + t.Fatalf("stat %s: %v", path, err) } } - return rootfs, kernel, initrd, modulesDir, packages -} - -func modelPathsLayoutForTest(dir string) paths.Layout { - return paths.Layout{ - ImagesDir: filepath.Join(dir, "images"), - } -} - -func TestStartVMDNSFailsWhenAddressBusy(t *testing.T) { - t.Parallel() - - packetConn, err := net.ListenPacket("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ListenPacket: %v", err) - } - defer packetConn.Close() - - d := &Daemon{} - if err := d.startVMDNS(packetConn.LocalAddr().String()); err == nil { - t.Fatal("startVMDNS() succeeded on occupied address, want failure") - } -} - -func TestSetDNSPublishesIntoDaemonServer(t *testing.T) { - t.Parallel() - - d := &Daemon{} - if err := d.startVMDNS("127.0.0.1:0"); err != nil { - t.Fatalf("startVMDNS: %v", err) - } - defer d.stopVMDNS() - - if err := d.setDNS(context.Background(), "devbox", "172.16.0.8"); err != nil { - t.Fatalf("setDNS: %v", err) - } - if _, ok := d.vmDNS.Lookup("devbox.vm"); !ok { - t.Fatal("devbox.vm missing after setDNS") - } -} - -func TestDispatchUsesPassedContext(t *testing.T) { - t.Parallel() - - db := openDefaultImageStore(t, t.TempDir()) - d := &Daemon{store: db} - ctx, cancel := context.WithCancel(context.Background()) - cancel() - - resp := d.dispatch(ctx, rpc.Request{ - Version: rpc.Version, - Method: "vm.list", - Params: mustJSON(t, api.Empty{}), - }) - - if resp.OK { - t.Fatal("dispatch() succeeded with canceled context") - } - if resp.Error == nil || !strings.Contains(resp.Error.Message, context.Canceled.Error()) { - t.Fatalf("dispatch() error = %+v, want context canceled", resp.Error) - } -} - -func TestHandleConnCancelsRequestWhenClientDisconnects(t *testing.T) { - t.Parallel() - - server, client := net.Pipe() - defer client.Close() - - requestCanceled := make(chan struct{}) - done := make(chan struct{}) - d := &Daemon{ - closing: make(chan struct{}), - requestHandler: func(ctx context.Context, req rpc.Request) rpc.Response { - if req.Method != "block" { - t.Errorf("request method = %q, want block", req.Method) - } - <-ctx.Done() - close(requestCanceled) - return rpc.NewError("operation_failed", ctx.Err().Error()) - }, - } - - go func() { - d.handleConn(server) - close(done) - }() - - if err := json.NewEncoder(client).Encode(rpc.Request{Version: rpc.Version, Method: "block"}); err != nil { - t.Fatalf("encode request: %v", err) - } - if err := client.Close(); err != nil { - t.Fatalf("close client: %v", err) - } - - select { - case <-requestCanceled: - case <-time.After(2 * time.Second): - t.Fatal("request context was not canceled after client disconnect") - } - - select { - case <-done: - case <-time.After(2 * time.Second): - t.Fatal("handleConn did not return after client disconnect") - } -} - -func TestWatchRequestDisconnectCancelsContextOnEOF(t *testing.T) { - t.Parallel() - - server, client := net.Pipe() - defer server.Close() - - reader := bufio.NewReader(server) - ctx, cancel := context.WithCancel(context.Background()) - t.Cleanup(cancel) - - d := &Daemon{closing: make(chan struct{})} - stop := d.watchRequestDisconnect(server, reader, "block", cancel) - defer stop() - - if err := client.Close(); err != nil { - t.Fatalf("close client: %v", err) - } - - select { - case <-ctx.Done(): - if !strings.Contains(ctx.Err().Error(), context.Canceled.Error()) { - t.Fatalf("ctx.Err() = %v, want canceled", ctx.Err()) - } - case <-time.After(2 * time.Second): - t.Fatal("watchRequestDisconnect did not cancel context") - } -} - -func mustJSON(t *testing.T, v any) []byte { - t.Helper() - data, err := json.Marshal(v) - if err != nil { - t.Fatalf("json.Marshal(%T): %v", v, err) - } - return data } diff --git a/internal/daemon/doctor.go b/internal/daemon/doctor.go index e4c9802..b29c312 100644 --- a/internal/daemon/doctor.go +++ b/internal/daemon/doctor.go @@ -2,12 +2,13 @@ package daemon import ( "context" - "fmt" + "database/sql" "strings" "banger/internal/config" "banger/internal/model" "banger/internal/paths" + "banger/internal/store" "banger/internal/system" ) @@ -25,34 +26,49 @@ func Doctor(ctx context.Context) (system.Report, error) { config: cfg, runner: system.NewRunner(), } + db, err := store.Open(layout.DBPath) + if err == nil { + defer db.Close() + d.store = db + } return d.doctorReport(ctx), nil } func (d *Daemon) doctorReport(ctx context.Context) system.Report { report := system.Report{} - report.AddPreflight("runtime bundle", d.runtimeBundleChecks(), runtimeBundleStatus(d.config)) + report.AddPreflight("host runtime", d.runtimeChecks(), runtimeStatus(d.config)) report.AddPreflight("core vm lifecycle", d.coreVMLifecycleChecks(), "required host tools available") - report.AddPreflight("vsock guest agent", d.vsockChecks(), "vsock agent prerequisites available") + report.AddPreflight("vsock guest agent", d.vsockChecks(), "vsock guest agent prerequisites available") d.addCapabilityDoctorChecks(ctx, &report) report.AddPreflight("image build", d.imageBuildChecks(ctx), "image build prerequisites available") return report } -func (d *Daemon) runtimeBundleChecks() *system.Preflight { +func (d *Daemon) runtimeChecks() *system.Preflight { checks := system.NewPreflight() - hint := paths.RuntimeBundleHint() - checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", hint) - checks.RequireFile(d.config.SSHKeyPath, "runtime ssh private key", `refresh the runtime bundle`) - checks.RequireExecutable(d.config.VSockAgentPath, "vsock agent", `run 'make build' or refresh the runtime bundle`) - checks.RequireFile(d.config.DefaultRootfs, "default rootfs image", `set "default_rootfs" or refresh the runtime bundle`) - checks.RequireFile(d.config.DefaultKernel, "kernel image", `set "default_kernel" or refresh the runtime bundle`) - if strings.TrimSpace(d.config.DefaultInitrd) != "" { - checks.RequireFile(d.config.DefaultInitrd, "initrd image", `set "default_initrd" or refresh the runtime bundle`) + checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", `install firecracker or set "firecracker_bin"`) + checks.RequireFile(d.config.SSHKeyPath, "ssh private key", `set "ssh_key_path" or let banger create its default key`) + if helper, err := d.vsockAgentBinary(); err == nil { + checks.RequireExecutable(helper, "vsock agent helper", `run 'make build' or reinstall banger`) + } else { + checks.Addf("%v", err) } - if strings.TrimSpace(d.config.DefaultPackagesFile) != "" { - checks.RequireFile(d.config.DefaultPackagesFile, "package manifest", `set "default_packages_file" or refresh the runtime bundle`) + if d.store != nil && strings.TrimSpace(d.config.DefaultImageName) != "" { + image, err := d.store.GetImageByName(context.Background(), d.config.DefaultImageName) + switch { + case err == nil: + checks.RequireFile(image.RootfsPath, "default image rootfs", `re-register or rebuild the default image`) + checks.RequireFile(image.KernelPath, "default image kernel", `re-register or rebuild the default image`) + if strings.TrimSpace(image.InitrdPath) != "" { + checks.RequireFile(image.InitrdPath, "default image initrd", `re-register or rebuild the default image`) + } + case err != nil && err != sql.ErrNoRows: + checks.Addf("failed to inspect default image %q: %v", d.config.DefaultImageName, err) + default: + checks.Addf("default image %q is not registered", d.config.DefaultImageName) + } } return checks } @@ -65,37 +81,33 @@ func (d *Daemon) coreVMLifecycleChecks() *system.Preflight { func (d *Daemon) imageBuildChecks(ctx context.Context) *system.Preflight { checks := system.NewPreflight() - d.addImageBuildPrereqs( - ctx, - checks, - firstNonEmpty(d.config.DefaultBaseRootfs, d.config.DefaultRootfs), - d.config.DefaultKernel, - d.config.DefaultInitrd, - d.config.DefaultModulesDir, - "", - ) + if d.store == nil || strings.TrimSpace(d.config.DefaultImageName) == "" { + checks.Addf("default image is not available for build inheritance") + return checks + } + image, err := d.store.GetImageByName(ctx, d.config.DefaultImageName) + if err != nil { + checks.Addf("default image %q is not registered", d.config.DefaultImageName) + return checks + } + d.addImageBuildPrereqs(ctx, checks, image.RootfsPath, image.KernelPath, image.InitrdPath, image.ModulesDir, "") return checks } func (d *Daemon) vsockChecks() *system.Preflight { checks := system.NewPreflight() - checks.RequireExecutable(d.config.VSockAgentPath, "vsock agent", `run 'make build' or refresh the runtime bundle`) + if helper, err := d.vsockAgentBinary(); err == nil { + checks.RequireExecutable(helper, "vsock agent helper", `run 'make build' or reinstall banger`) + } else { + checks.Addf("%v", err) + } checks.RequireFile(vsockHostDevicePath, "vsock host device", "load the vhost_vsock kernel module on the host") return checks } -func runtimeBundleStatus(cfg model.DaemonConfig) string { - if strings.TrimSpace(cfg.RuntimeDir) == "" { - return "runtime dir not configured" +func runtimeStatus(cfg model.DaemonConfig) string { + if strings.TrimSpace(cfg.FirecrackerBin) == "" { + return "firecracker not configured" } - return fmt.Sprintf("runtime dir %s", cfg.RuntimeDir) -} - -func firstNonEmpty(values ...string) string { - for _, value := range values { - if strings.TrimSpace(value) != "" { - return value - } - } - return "" + return "firecracker and ssh key resolved" } diff --git a/internal/daemon/imagebuild.go b/internal/daemon/imagebuild.go index bccf6f3..f22f040 100644 --- a/internal/daemon/imagebuild.go +++ b/internal/daemon/imagebuild.go @@ -3,7 +3,6 @@ package daemon import ( "bytes" "context" - "crypto/sha256" "errors" "fmt" "io" @@ -16,6 +15,7 @@ import ( "banger/internal/guest" "banger/internal/guestnet" "banger/internal/hostnat" + "banger/internal/imagepreset" "banger/internal/model" "banger/internal/opencode" "banger/internal/system" @@ -39,13 +39,13 @@ const ( type imageBuildSpec struct { ID string Name string - BaseRootfs string + SourceRootfs string RootfsPath string BuildLog io.Writer KernelPath string InitrdPath string ModulesDir string - PackagesPath string + Packages []string InstallDocker bool Size string } @@ -66,15 +66,11 @@ func (d *Daemon) runImageBuild(ctx context.Context, spec imageBuildSpec) error { } func (d *Daemon) runImageBuildNative(ctx context.Context, spec imageBuildSpec) (err error) { - packages, err := system.ReadNormalizedLines(spec.PackagesPath) - if err != nil { - return err - } - if err := system.CopyFilePreferClone(spec.BaseRootfs, spec.RootfsPath); err != nil { + if err := system.CopyFilePreferClone(spec.SourceRootfs, spec.RootfsPath); err != nil { return err } if spec.Size != "" { - if err := resizeRootfs(spec.BaseRootfs, spec.RootfsPath, spec.Size); err != nil { + if err := resizeRootfs(spec.SourceRootfs, spec.RootfsPath, spec.Size); err != nil { return err } } @@ -110,7 +106,11 @@ func (d *Daemon) runImageBuildNative(ctx context.Context, spec imageBuildSpec) ( return err } - helperBytes, err := os.ReadFile(d.config.VSockAgentPath) + vsockAgentPath, err := d.vsockAgentBinary() + if err != nil { + return err + } + helperBytes, err := os.ReadFile(vsockAgentPath) if err != nil { return err } @@ -123,7 +123,7 @@ func (d *Daemon) runImageBuildNative(ctx context.Context, spec imageBuildSpec) ( if err := writeBuildLog(spec.BuildLog, "configuring guest"); err != nil { return err } - if err := client.RunScript(ctx, buildProvisionScript(vm.Name, d.config.DefaultDNS, string(authorizedKey), packages, spec.InstallDocker), spec.BuildLog); err != nil { + if err := client.RunScript(ctx, buildProvisionScript(vm.Name, d.config.DefaultDNS, string(authorizedKey), spec.Packages, spec.InstallDocker), spec.BuildLog); err != nil { return err } if strings.TrimSpace(spec.ModulesDir) != "" { @@ -428,6 +428,5 @@ func writeBuildLog(w io.Writer, message string) error { } func packagesHash(lines []string) string { - sum := sha256.Sum256([]byte(strings.Join(lines, "\n") + "\n")) - return fmt.Sprintf("%x", sum) + return imagepreset.Hash(lines) } diff --git a/internal/daemon/images.go b/internal/daemon/images.go index 365e53d..b20873e 100644 --- a/internal/daemon/images.go +++ b/internal/daemon/images.go @@ -10,8 +10,8 @@ import ( "strings" "banger/internal/api" + "banger/internal/imagepreset" "banger/internal/model" - "banger/internal/paths" "banger/internal/system" ) @@ -37,12 +37,13 @@ func (d *Daemon) BuildImage(ctx context.Context, params api.ImageBuildParams) (i if _, err := d.FindImage(ctx, name); err == nil { return model.Image{}, fmt.Errorf("image name already exists: %s", name) } - baseRootfs := params.BaseRootfs - if baseRootfs == "" { - baseRootfs = d.config.DefaultBaseRootfs + fromImage := strings.TrimSpace(params.FromImage) + if fromImage == "" { + return model.Image{}, fmt.Errorf("from-image is required") } - if baseRootfs == "" { - return model.Image{}, fmt.Errorf("base rootfs is required; %s", paths.RuntimeBundleHint()) + baseImage, err := d.FindImage(ctx, fromImage) + if err != nil { + return model.Image{}, err } id, err := model.NewID() if err != nil { @@ -50,9 +51,6 @@ func (d *Daemon) BuildImage(ctx context.Context, params api.ImageBuildParams) (i } now := model.Now() artifactDir := filepath.Join(d.layout.ImagesDir, id) - if err := os.MkdirAll(artifactDir, 0o755); err != nil { - return model.Image{}, err - } buildLogDir := filepath.Join(d.layout.StateDir, "image-build") if err := os.MkdirAll(buildLogDir, 0o755); err != nil { return model.Image{}, err @@ -64,73 +62,80 @@ func (d *Daemon) BuildImage(ctx context.Context, params api.ImageBuildParams) (i return model.Image{}, err } defer logFile.Close() - rootfsPath := filepath.Join(artifactDir, "rootfs.ext4") - workSeedPath := filepath.Join(artifactDir, "work-seed.ext4") - kernelPath := params.KernelPath - if kernelPath == "" { - kernelPath = d.config.DefaultKernel - } - initrdPath := params.InitrdPath - if initrdPath == "" { - initrdPath = d.config.DefaultInitrd - } - modulesDir := params.ModulesDir - if modulesDir == "" { - modulesDir = d.config.DefaultModulesDir - } - if err := d.validateImageBuildPrereqs(ctx, baseRootfs, kernelPath, initrdPath, modulesDir, params.Size); err != nil { + stageDir, err := os.MkdirTemp(d.layout.ImagesDir, id+".build-") + if err != nil { return model.Image{}, err } + cleanupStage := true + defer func() { + if cleanupStage { + _ = os.RemoveAll(stageDir) + } + }() + rootfsPath := filepath.Join(stageDir, "rootfs.ext4") + workSeedPath := filepath.Join(stageDir, "work-seed.ext4") + kernelSource := firstNonEmpty(params.KernelPath, baseImage.KernelPath) + initrdSource := firstNonEmpty(params.InitrdPath, baseImage.InitrdPath) + modulesSource := firstNonEmpty(params.ModulesDir, baseImage.ModulesDir) + if err := d.validateImageBuildPrereqs(ctx, baseImage.RootfsPath, kernelSource, initrdSource, modulesSource, params.Size); err != nil { + return model.Image{}, err + } + kernelPath, initrdPath, modulesDir, err := stageManagedBootArtifacts(ctx, d.runner, stageDir, kernelSource, initrdSource, modulesSource) + if err != nil { + return model.Image{}, err + } + packages := imagepreset.DebianBasePackages() + metadataPackages := imageBuildMetadataPackages(params.Docker) spec := imageBuildSpec{ ID: id, Name: name, - BaseRootfs: baseRootfs, + SourceRootfs: baseImage.RootfsPath, RootfsPath: rootfsPath, BuildLog: logFile, KernelPath: kernelPath, InitrdPath: initrdPath, ModulesDir: modulesDir, - PackagesPath: d.config.DefaultPackagesFile, + Packages: packages, InstallDocker: params.Docker, Size: params.Size, } - op.stage("launch_builder", "build_log_path", buildLogPath, "artifact_dir", artifactDir) + op.stage("launch_builder", "build_log_path", buildLogPath, "artifact_dir", artifactDir, "from_image", baseImage.Name) imageBuildStage(ctx, "launch_builder", "building rootfs from base image") if err := d.runImageBuild(ctx, spec); err != nil { _ = logFile.Sync() - _ = os.RemoveAll(artifactDir) return model.Image{}, err } imageBuildStage(ctx, "prepare_work_seed", "building reusable work seed") if err := system.BuildWorkSeedImage(ctx, d.runner, rootfsPath, workSeedPath); err != nil { _ = logFile.Sync() - _ = os.RemoveAll(artifactDir) return model.Image{}, err } imageBuildStage(ctx, "seed_ssh", "seeding runtime SSH access") seededSSHPublicKeyFingerprint, err := d.seedAuthorizedKeyOnExt4Image(ctx, workSeedPath) if err != nil { _ = logFile.Sync() - _ = os.RemoveAll(artifactDir) return model.Image{}, err } imageBuildStage(ctx, "write_metadata", "writing image metadata") - if err := writePackagesMetadata(rootfsPath, d.config.DefaultPackagesFile); err != nil { + if err := writePackagesMetadata(rootfsPath, metadataPackages); err != nil { _ = logFile.Sync() - _ = os.RemoveAll(artifactDir) return model.Image{}, err } + op.stage("activate_artifacts", "artifact_dir", artifactDir) + if err := os.Rename(stageDir, artifactDir); err != nil { + return model.Image{}, err + } + cleanupStage = false image = model.Image{ ID: id, Name: name, Managed: true, ArtifactDir: artifactDir, - RootfsPath: rootfsPath, - WorkSeedPath: workSeedPath, - KernelPath: kernelPath, - InitrdPath: initrdPath, - ModulesDir: modulesDir, - PackagesPath: d.config.DefaultPackagesFile, + RootfsPath: filepath.Join(artifactDir, "rootfs.ext4"), + WorkSeedPath: filepath.Join(artifactDir, "work-seed.ext4"), + KernelPath: filepath.Join(artifactDir, "kernel"), + InitrdPath: stageOptionalArtifactPath(artifactDir, initrdPath, "initrd.img"), + ModulesDir: stageOptionalArtifactPath(artifactDir, modulesDir, "modules"), BuildSize: params.Size, SeededSSHPublicKeyFingerprint: seededSSHPublicKeyFingerprint, Docker: params.Docker, @@ -174,19 +179,12 @@ func (d *Daemon) RegisterImage(ctx context.Context, params api.ImageRegisterPara } kernelPath := strings.TrimSpace(params.KernelPath) if kernelPath == "" { - kernelPath = d.config.DefaultKernel + return model.Image{}, fmt.Errorf("kernel path is required") } initrdPath := strings.TrimSpace(params.InitrdPath) - if initrdPath == "" { - initrdPath = d.config.DefaultInitrd - } modulesDir := strings.TrimSpace(params.ModulesDir) - if modulesDir == "" { - modulesDir = d.config.DefaultModulesDir - } - packagesPath := strings.TrimSpace(params.PackagesPath) - if err := validateImageRegisterPaths(rootfsPath, workSeedPath, kernelPath, initrdPath, modulesDir, packagesPath); err != nil { + if err := validateImageRegisterPaths(rootfsPath, workSeedPath, kernelPath, initrdPath, modulesDir); err != nil { return model.Image{}, err } @@ -203,7 +201,6 @@ func (d *Daemon) RegisterImage(ctx context.Context, params api.ImageRegisterPara image.KernelPath = kernelPath image.InitrdPath = initrdPath image.ModulesDir = modulesDir - image.PackagesPath = packagesPath image.Docker = params.Docker image.UpdatedAt = now case errors.Is(lookupErr, sql.ErrNoRows): @@ -220,7 +217,6 @@ func (d *Daemon) RegisterImage(ctx context.Context, params api.ImageRegisterPara KernelPath: kernelPath, InitrdPath: initrdPath, ModulesDir: modulesDir, - PackagesPath: packagesPath, Docker: params.Docker, CreatedAt: now, UpdatedAt: now, @@ -255,7 +251,7 @@ func (d *Daemon) PromoteImage(ctx context.Context, idOrName string) (image model if image.Managed { return model.Image{}, fmt.Errorf("image %s is already managed", image.Name) } - if err := validateImagePromotePaths(image.RootfsPath, image.KernelPath, image.InitrdPath, image.ModulesDir, image.PackagesPath); err != nil { + if err := validateImagePromotePaths(image.RootfsPath, image.KernelPath, image.InitrdPath, image.ModulesDir); err != nil { return model.Image{}, err } if strings.TrimSpace(d.layout.ImagesDir) == "" { @@ -313,6 +309,10 @@ func (d *Daemon) PromoteImage(ctx context.Context, idOrName string) (image model } else { image.SeededSSHPublicKeyFingerprint = "" } + _, initrdPath, modulesDir, err := stageManagedBootArtifacts(ctx, d.runner, stageDir, image.KernelPath, image.InitrdPath, image.ModulesDir) + if err != nil { + return model.Image{}, err + } op.stage("activate_artifacts", "artifact_dir", artifactDir) if err := os.Rename(stageDir, artifactDir); err != nil { @@ -326,6 +326,9 @@ func (d *Daemon) PromoteImage(ctx context.Context, idOrName string) (image model if workSeedPath != "" { image.WorkSeedPath = filepath.Join(artifactDir, "work-seed.ext4") } + image.KernelPath = filepath.Join(artifactDir, "kernel") + image.InitrdPath = stageOptionalArtifactPath(artifactDir, initrdPath, "initrd.img") + image.ModulesDir = stageOptionalArtifactPath(artifactDir, modulesDir, "modules") image.UpdatedAt = model.Now() if err := d.store.UpsertImage(ctx, image); err != nil { _ = os.RemoveAll(artifactDir) @@ -334,26 +337,23 @@ func (d *Daemon) PromoteImage(ctx context.Context, idOrName string) (image model return image, nil } -func validateImageRegisterPaths(rootfsPath, workSeedPath, kernelPath, initrdPath, modulesDir, packagesPath string) error { +func validateImageRegisterPaths(rootfsPath, workSeedPath, kernelPath, initrdPath, modulesDir string) error { checks := system.NewPreflight() checks.RequireFile(rootfsPath, "rootfs image", `pass --rootfs `) - checks.RequireFile(kernelPath, "kernel image", `pass --kernel or set "default_kernel"`) + checks.RequireFile(kernelPath, "kernel image", `pass --kernel `) if workSeedPath != "" { checks.RequireFile(workSeedPath, "work-seed image", `pass --work-seed or rebuild the image with a work seed`) } if initrdPath != "" { - checks.RequireFile(initrdPath, "initrd image", `pass --initrd or set "default_initrd"`) + checks.RequireFile(initrdPath, "initrd image", `pass --initrd `) } if modulesDir != "" { - checks.RequireDir(modulesDir, "kernel modules dir", `pass --modules or set "default_modules_dir"`) - } - if packagesPath != "" { - checks.RequireFile(packagesPath, "packages manifest", `pass --packages `) + checks.RequireDir(modulesDir, "kernel modules dir", `pass --modules `) } return checks.Err("image register failed") } -func validateImagePromotePaths(rootfsPath, kernelPath, initrdPath, modulesDir, packagesPath string) error { +func validateImagePromotePaths(rootfsPath, kernelPath, initrdPath, modulesDir string) error { checks := system.NewPreflight() checks.RequireFile(rootfsPath, "rootfs image", `re-register the image with a valid rootfs`) checks.RequireFile(kernelPath, "kernel image", `re-register the image with a valid kernel`) @@ -363,22 +363,15 @@ func validateImagePromotePaths(rootfsPath, kernelPath, initrdPath, modulesDir, p if modulesDir != "" { checks.RequireDir(modulesDir, "kernel modules dir", `re-register the image with a valid modules dir`) } - if packagesPath != "" { - checks.RequireFile(packagesPath, "packages manifest", `re-register the image with a valid packages manifest`) - } return checks.Err("image promote failed") } -func writePackagesMetadata(rootfsPath, packagesPath string) error { - if rootfsPath == "" || packagesPath == "" { +func writePackagesMetadata(rootfsPath string, packages []string) error { + if rootfsPath == "" || len(packages) == 0 { return nil } - lines, err := system.ReadNormalizedLines(packagesPath) - if err != nil { - return err - } metadataPath := rootfsPath + ".packages.sha256" - return os.WriteFile(metadataPath, []byte(packagesHash(lines)+"\n"), 0o644) + return os.WriteFile(metadataPath, []byte(packagesHash(packages)+"\n"), 0o644) } func (d *Daemon) DeleteImage(ctx context.Context, idOrName string) (model.Image, error) { @@ -406,3 +399,52 @@ func (d *Daemon) DeleteImage(ctx context.Context, idOrName string) (model.Image, } return image, nil } + +func stageManagedBootArtifacts(ctx context.Context, runner system.CommandRunner, artifactDir, kernelSource, initrdSource, modulesSource string) (string, string, string, error) { + kernelPath := filepath.Join(artifactDir, "kernel") + if err := system.CopyFilePreferClone(kernelSource, kernelPath); err != nil { + return "", "", "", err + } + initrdPath := "" + if strings.TrimSpace(initrdSource) != "" { + initrdPath = filepath.Join(artifactDir, "initrd.img") + if err := system.CopyFilePreferClone(initrdSource, initrdPath); err != nil { + return "", "", "", err + } + } + modulesDir := "" + if strings.TrimSpace(modulesSource) != "" { + modulesDir = filepath.Join(artifactDir, "modules") + if err := os.MkdirAll(modulesDir, 0o755); err != nil { + return "", "", "", err + } + if err := system.CopyDirContents(ctx, runner, modulesSource, modulesDir, false); err != nil { + return "", "", "", err + } + } + return kernelPath, initrdPath, modulesDir, nil +} + +func imageBuildMetadataPackages(docker bool) []string { + packages := imagepreset.DebianBasePackages() + if docker { + packages = append(packages, "#feature:docker") + } + return packages +} + +func stageOptionalArtifactPath(artifactDir, stagedPath, name string) string { + if strings.TrimSpace(stagedPath) == "" { + return "" + } + return filepath.Join(artifactDir, name) +} + +func firstNonEmpty(values ...string) string { + for _, value := range values { + if strings.TrimSpace(value) != "" { + return value + } + } + return "" +} diff --git a/internal/daemon/logger_test.go b/internal/daemon/logger_test.go index d848064..4ad9e29 100644 --- a/internal/daemon/logger_test.go +++ b/internal/daemon/logger_test.go @@ -69,6 +69,7 @@ func TestStartVMLockedLogsBridgeFailure(t *testing.T) { if err := os.WriteFile(vsockHelper, []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { t.Fatalf("write vsock helper: %v", err) } + t.Setenv("BANGER_VSOCK_AGENT_BIN", vsockHelper) rootfsPath := filepath.Join(t.TempDir(), "rootfs.ext4") kernelPath := filepath.Join(t.TempDir(), "vmlinux") for _, path := range []string{rootfsPath, kernelPath} { @@ -109,7 +110,6 @@ func TestStartVMLockedLogsBridgeFailure(t *testing.T) { BridgeIP: model.DefaultBridgeIP, DefaultDNS: model.DefaultDNS, FirecrackerBin: firecrackerBin, - VSockAgentPath: vsockHelper, StatsPollInterval: model.DefaultStatsPollInterval, }, runner: runner, @@ -148,11 +148,10 @@ func TestBuildImagePreservesBuildLogOnFailure(t *testing.T) { baseRootfs := filepath.Join(t.TempDir(), "base.ext4") kernelPath := filepath.Join(t.TempDir(), "vmlinux") - packagesPath := filepath.Join(t.TempDir(), "packages.apt") sshKeyPath := filepath.Join(t.TempDir(), "id_ed25519") firecrackerBin := filepath.Join(t.TempDir(), "firecracker") vsockHelper := filepath.Join(t.TempDir(), "banger-vsock-agent") - for _, path := range []string{baseRootfs, kernelPath, packagesPath, sshKeyPath} { + for _, path := range []string{baseRootfs, kernelPath, sshKeyPath} { if err := os.WriteFile(path, []byte("artifact"), 0o644); err != nil { t.Fatalf("write %s: %v", path, err) } @@ -160,6 +159,7 @@ func TestBuildImagePreservesBuildLogOnFailure(t *testing.T) { if err := os.WriteFile(vsockHelper, []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { t.Fatalf("write %s: %v", vsockHelper, err) } + t.Setenv("BANGER_VSOCK_AGENT_BIN", vsockHelper) if err := os.WriteFile(firecrackerBin, []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { t.Fatalf("write %s: %v", firecrackerBin, err) } @@ -175,18 +175,26 @@ func TestBuildImagePreservesBuildLogOnFailure(t *testing.T) { if err != nil { t.Fatalf("newDaemonLogger: %v", err) } + baseImage := model.Image{ + ID: "base-image", + Name: "base-image", + RootfsPath: baseRootfs, + KernelPath: kernelPath, + CreatedAt: model.Now(), + UpdatedAt: model.Now(), + } + if err := store.UpsertImage(ctx, baseImage); err != nil { + t.Fatalf("UpsertImage(base): %v", err) + } d := &Daemon{ layout: paths.Layout{ StateDir: stateDir, ImagesDir: imagesDir, }, config: model.DaemonConfig{ - RuntimeDir: t.TempDir(), - DefaultImageName: "default", - DefaultPackagesFile: packagesPath, - SSHKeyPath: sshKeyPath, - FirecrackerBin: firecrackerBin, - VSockAgentPath: vsockHelper, + DefaultImageName: "base-image", + SSHKeyPath: sshKeyPath, + FirecrackerBin: firecrackerBin, }, store: store, runner: runner, @@ -195,7 +203,7 @@ func TestBuildImagePreservesBuildLogOnFailure(t *testing.T) { if _, err := fmt.Fprintln(spec.BuildLog, "builder-stdout"); err != nil { return err } - if spec.BaseRootfs != baseRootfs || spec.KernelPath != kernelPath || spec.PackagesPath != packagesPath { + if spec.SourceRootfs != baseRootfs || spec.KernelPath == kernelPath || len(spec.Packages) == 0 { t.Fatalf("unexpected image build spec: %+v", spec) } return errors.New("builder failed") @@ -204,7 +212,7 @@ func TestBuildImagePreservesBuildLogOnFailure(t *testing.T) { _, err = d.BuildImage(ctx, api.ImageBuildParams{ Name: "broken-image", - BaseRootfs: baseRootfs, + FromImage: baseImage.Name, KernelPath: kernelPath, }) if err == nil || !strings.Contains(err.Error(), "inspect ") { diff --git a/internal/daemon/preflight.go b/internal/daemon/preflight.go index c4dd41c..0d3c251 100644 --- a/internal/daemon/preflight.go +++ b/internal/daemon/preflight.go @@ -5,7 +5,6 @@ import ( "strings" "banger/internal/model" - "banger/internal/paths" "banger/internal/system" ) @@ -50,16 +49,18 @@ func (d *Daemon) addNATPrereqs(ctx context.Context, checks *system.Preflight) { } func (d *Daemon) addBaseStartPrereqs(checks *system.Preflight, image model.Image) { - hint := paths.RuntimeBundleHint() - d.addBaseStartCommandPrereqs(checks) - checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", hint) - checks.RequireExecutable(d.config.VSockAgentPath, "vsock agent", `run 'make build' or refresh the runtime bundle`) + checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", `install firecracker or set "firecracker_bin"`) + if helper, err := d.vsockAgentBinary(); err == nil { + checks.RequireExecutable(helper, "vsock agent helper", `run 'make build' or reinstall banger`) + } else { + checks.Addf("%v", err) + } checks.RequireFile(vsockHostDevicePath, "vsock host device", "load the vhost_vsock kernel module on the host") - checks.RequireFile(image.RootfsPath, "rootfs image", "select a valid image or rebuild the runtime bundle") - checks.RequireFile(image.KernelPath, "kernel image", `set "default_kernel" or refresh the runtime bundle`) + checks.RequireFile(image.RootfsPath, "rootfs image", "select a valid registered image") + checks.RequireFile(image.KernelPath, "kernel image", `re-register or rebuild the image with a valid kernel`) if strings.TrimSpace(image.InitrdPath) != "" { - checks.RequireFile(image.InitrdPath, "initrd image", `set "default_initrd" or refresh the runtime bundle`) + checks.RequireFile(image.InitrdPath, "initrd image", `re-register or rebuild the image with a valid initrd`) } } @@ -70,30 +71,26 @@ func (d *Daemon) addBaseStartCommandPrereqs(checks *system.Preflight) { } func (d *Daemon) addImageBuildPrereqs(ctx context.Context, checks *system.Preflight, baseRootfs, kernelPath, initrdPath, modulesDir, sizeSpec string) { - hint := paths.RuntimeBundleHint() - for _, command := range []string{"sudo", "ip", "pgrep", "chown", "chmod", "kill"} { checks.RequireCommand(command, toolHint(command)) } for _, command := range []string{"mkfs.ext4", "mount", "umount", "cp"} { checks.RequireCommand(command, toolHint(command)) } - checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", hint) - checks.RequireFile(d.config.SSHKeyPath, "runtime ssh private key", `refresh the runtime bundle`) - checks.RequireExecutable(d.config.VSockAgentPath, "vsock agent", `run 'make build' or refresh the runtime bundle`) - checks.RequireFile(baseRootfs, "base rootfs image", `pass --base-rootfs or set "default_base_rootfs"`) - checks.RequireFile(kernelPath, "kernel image", `pass --kernel or set "default_kernel"`) - checks.RequireFile(d.config.DefaultPackagesFile, "package manifest", `set "default_packages_file" or refresh the runtime bundle`) + checks.RequireExecutable(d.config.FirecrackerBin, "firecracker binary", `install firecracker or set "firecracker_bin"`) + checks.RequireFile(d.config.SSHKeyPath, "ssh private key", `set "ssh_key_path" or let banger create its default key`) + if helper, err := d.vsockAgentBinary(); err == nil { + checks.RequireExecutable(helper, "vsock agent helper", `run 'make build' or reinstall banger`) + } else { + checks.Addf("%v", err) + } + checks.RequireFile(baseRootfs, "base image rootfs", `pass --from-image with a valid registered image`) + checks.RequireFile(kernelPath, "kernel image", `pass --kernel or build from an image with a valid kernel`) if strings.TrimSpace(initrdPath) != "" { - checks.RequireFile(initrdPath, "initrd image", `pass --initrd or set "default_initrd"`) + checks.RequireFile(initrdPath, "initrd image", `pass --initrd or build from an image with a valid initrd`) } if strings.TrimSpace(modulesDir) != "" { - checks.RequireDir(modulesDir, "modules directory", `pass --modules or set "default_modules_dir"`) - } - if strings.TrimSpace(d.config.DefaultPackagesFile) != "" { - if _, err := system.ReadNormalizedLines(d.config.DefaultPackagesFile); err != nil { - checks.Addf("package manifest at %s is invalid: %v", d.config.DefaultPackagesFile, err) - } + checks.RequireDir(modulesDir, "modules directory", `pass --modules or build from an image with a valid modules dir`) } if strings.TrimSpace(sizeSpec) != "" { checks.RequireCommand("e2fsck", toolHint("e2fsck")) diff --git a/internal/daemon/runtime_assets.go b/internal/daemon/runtime_assets.go new file mode 100644 index 0000000..16c4cf6 --- /dev/null +++ b/internal/daemon/runtime_assets.go @@ -0,0 +1,15 @@ +package daemon + +import ( + "fmt" + + "banger/internal/paths" +) + +func (d *Daemon) vsockAgentBinary() (string, error) { + path, err := paths.CompanionBinaryPath("banger-vsock-agent") + if err != nil { + return "", fmt.Errorf("vsock agent helper not available: %w", err) + } + return path, nil +} diff --git a/internal/daemon/vm.go b/internal/daemon/vm.go index a602f1a..251f039 100644 --- a/internal/daemon/vm.go +++ b/internal/daemon/vm.go @@ -16,7 +16,7 @@ import ( "banger/internal/guest" "banger/internal/guestconfig" "banger/internal/model" - "banger/internal/paths" + "banger/internal/namegen" "banger/internal/system" "banger/internal/vmdns" "banger/internal/vsockagent" @@ -998,13 +998,20 @@ func (d *Daemon) createTap(ctx context.Context, tap string) error { func (d *Daemon) firecrackerBinary() (string, error) { if d.config.FirecrackerBin == "" { - return "", fmt.Errorf("firecracker binary not configured; %s", paths.RuntimeBundleHint()) + return "", fmt.Errorf("firecracker binary not configured; install firecracker or set firecracker_bin") } path := d.config.FirecrackerBin - if !exists(path) { - return "", fmt.Errorf("firecracker binary not found at %s; %s", path, paths.RuntimeBundleHint()) + if strings.ContainsRune(path, os.PathSeparator) { + if !exists(path) { + return "", fmt.Errorf("firecracker binary not found at %s; install firecracker or set firecracker_bin", path) + } + return path, nil } - return path, nil + resolved, err := system.LookupExecutable(path) + if err != nil { + return "", fmt.Errorf("firecracker binary %q not found in PATH; install firecracker or set firecracker_bin", path) + } + return resolved, nil } func (d *Daemon) ensureSocketAccess(ctx context.Context, socketPath, label string) error { @@ -1190,14 +1197,9 @@ func (d *Daemon) killVMProcess(ctx context.Context, pid int) error { } func (d *Daemon) generateName(ctx context.Context) (string, error) { - if exists(d.config.NamegenPath) { - out, err := d.runner.Run(ctx, d.config.NamegenPath) - if err == nil { - name := strings.TrimSpace(string(out)) - if name != "" { - return name, nil - } - } + _ = ctx + if name := strings.TrimSpace(namegen.Generate()); name != "" { + return name, nil } return "vm-" + strconv.FormatInt(time.Now().Unix(), 10), nil } diff --git a/internal/daemon/vm_test.go b/internal/daemon/vm_test.go index 3298a69..74c1881 100644 --- a/internal/daemon/vm_test.go +++ b/internal/daemon/vm_test.go @@ -4,13 +4,14 @@ import ( "context" "crypto/rand" "crypto/rsa" + "crypto/tls" "crypto/x509" "encoding/pem" "errors" "fmt" + "math/big" "net" "net/http" - "net/http/httptest" "os" "os/exec" "path/filepath" @@ -183,6 +184,7 @@ func TestRebuildDNSIncludesOnlyLiveRunningVMs(t *testing.T) { server, err := vmdns.New("127.0.0.1:0", nil) if err != nil { + skipIfSocketRestricted(t, err) t.Fatalf("vmdns.New: %v", err) } t.Cleanup(func() { @@ -274,6 +276,7 @@ func TestHealthVMReturnsHealthyForRunningGuest(t *testing.T) { vsockSock := filepath.Join(t.TempDir(), "fc.vsock") listener, err := net.Listen("unix", vsockSock) if err != nil { + skipIfSocketRestricted(t, err) t.Fatalf("listen vsock: %v", err) } t.Cleanup(func() { @@ -367,6 +370,7 @@ func TestPingVMAliasReturnsAliveForHealthyVM(t *testing.T) { vsockSock := filepath.Join(t.TempDir(), "fc.vsock") listener, err := net.Listen("unix", vsockSock) if err != nil { + skipIfSocketRestricted(t, err) t.Fatalf("listen vsock: %v", err) } t.Cleanup(func() { @@ -441,32 +445,17 @@ func TestPortsVMReturnsEnrichedPortsAndWebSchemes(t *testing.T) { _ = fake.Wait() }) - webServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + webAddr := startHTTPServerOnTCP4(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) })) - t.Cleanup(webServer.Close) - webAddr, err := net.ResolveTCPAddr("tcp", strings.TrimPrefix(webServer.URL, "http://")) - if err != nil { - t.Fatalf("ResolveTCPAddr: %v", err) - } - tlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + tlsAddr := startHTTPSServerOnTCP4(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusAccepted) })) - tlsListener, err := net.Listen("tcp4", "127.0.0.1:0") - if err != nil { - t.Fatalf("listen tls: %v", err) - } - tlsServer.Listener = tlsListener - tlsServer.StartTLS() - t.Cleanup(tlsServer.Close) - tlsAddr, err := net.ResolveTCPAddr("tcp", strings.TrimPrefix(tlsServer.URL, "https://")) - if err != nil { - t.Fatalf("ResolveTCPAddr(tls): %v", err) - } vsockSock := filepath.Join(t.TempDir(), "fc.vsock") listener, err := net.Listen("unix", vsockSock) if err != nil { + skipIfSocketRestricted(t, err) t.Fatalf("listen vsock: %v", err) } t.Cleanup(func() { @@ -1263,6 +1252,7 @@ func startFakeFirecrackerAPI(t *testing.T, apiSock string) { } listener, err := net.Listen("unix", apiSock) if err != nil { + skipIfSocketRestricted(t, err) t.Fatalf("listen unix %s: %v", apiSock, err) } mux := http.NewServeMux() @@ -1283,6 +1273,72 @@ func startFakeFirecrackerAPI(t *testing.T, apiSock string) { }) } +func skipIfSocketRestricted(t *testing.T, err error) { + t.Helper() + if err == nil { + return + } + if strings.Contains(strings.ToLower(err.Error()), "operation not permitted") { + t.Skipf("socket creation is restricted in this environment: %v", err) + } +} + +func startHTTPServerOnTCP4(t *testing.T, handler http.Handler) *net.TCPAddr { + t.Helper() + listener, err := net.Listen("tcp4", "127.0.0.1:0") + if err != nil { + skipIfSocketRestricted(t, err) + t.Fatalf("listen http: %v", err) + } + server := &http.Server{Handler: handler} + go func() { + _ = server.Serve(listener) + }() + t.Cleanup(func() { + _ = server.Close() + }) + return listener.Addr().(*net.TCPAddr) +} + +func startHTTPSServerOnTCP4(t *testing.T, handler http.Handler) *net.TCPAddr { + t.Helper() + privateKey, err := rsa.GenerateKey(rand.Reader, 1024) + if err != nil { + t.Fatalf("GenerateKey: %v", err) + } + template := &x509.Certificate{ + SerialNumber: big.NewInt(1), + NotBefore: time.Now().Add(-time.Hour), + NotAfter: time.Now().Add(time.Hour), + IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}, + KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment, + ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, + } + der, err := x509.CreateCertificate(rand.Reader, template, template, &privateKey.PublicKey, privateKey) + if err != nil { + t.Fatalf("CreateCertificate: %v", err) + } + certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}) + keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(privateKey)}) + cert, err := tls.X509KeyPair(certPEM, keyPEM) + if err != nil { + t.Fatalf("X509KeyPair: %v", err) + } + listener, err := net.Listen("tcp4", "127.0.0.1:0") + if err != nil { + skipIfSocketRestricted(t, err) + t.Fatalf("listen https: %v", err) + } + server := &http.Server{Handler: handler} + go func() { + _ = server.Serve(tls.NewListener(listener, &tls.Config{Certificates: []tls.Certificate{cert}})) + }() + t.Cleanup(func() { + _ = server.Close() + }) + return listener.Addr().(*net.TCPAddr) +} + type processKillingRunner struct { *scriptedRunner proc *exec.Cmd diff --git a/internal/imagepreset/preset.go b/internal/imagepreset/preset.go new file mode 100644 index 0000000..d1de16e --- /dev/null +++ b/internal/imagepreset/preset.go @@ -0,0 +1,57 @@ +package imagepreset + +import ( + "crypto/sha256" + "fmt" + "strings" +) + +var debianBase = []string{ + "make", + "git", + "less", + "tree", + "ca-certificates", + "curl", + "wget", + "iproute2", + "vim", + "tmux", +} + +var voidBase = []string{ + "base-minimal", + "base-devel", + "bash", + "ca-certificates", + "curl", + "docker", + "docker-compose", + "e2fsprogs", + "git", + "iproute2", + "less", + "make", + "openssh", + "procps-ng", + "runit", + "shadow", + "sudo", + "tmux", + "tree", + "vim", + "wget", +} + +func DebianBasePackages() []string { + return append([]string(nil), debianBase...) +} + +func VoidBasePackages() []string { + return append([]string(nil), voidBase...) +} + +func Hash(lines []string) string { + sum := sha256.Sum256([]byte(strings.Join(lines, "\n") + "\n")) + return fmt.Sprintf("%x", sum) +} diff --git a/internal/model/types.go b/internal/model/types.go index 2955765..f986505 100644 --- a/internal/model/types.go +++ b/internal/model/types.go @@ -35,15 +35,10 @@ const ( ) type DaemonConfig struct { - RuntimeDir string LogLevel string WebListenAddr string FirecrackerBin string SSHKeyPath string - NamegenPath string - CustomizeScript string - VSockAgentPath string - DefaultWorkSeed string AutoStopStaleAfter time.Duration StatsPollInterval time.Duration MetricsPollInterval time.Duration @@ -53,12 +48,6 @@ type DaemonConfig struct { TapPoolSize int DefaultDNS string DefaultImageName string - DefaultRootfs string - DefaultBaseRootfs string - DefaultKernel string - DefaultInitrd string - DefaultModulesDir string - DefaultPackagesFile string } type Image struct { @@ -71,7 +60,6 @@ type Image struct { KernelPath string `json:"kernel_path"` InitrdPath string `json:"initrd_path,omitempty"` ModulesDir string `json:"modules_dir,omitempty"` - PackagesPath string `json:"packages_path,omitempty"` BuildSize string `json:"build_size,omitempty"` SeededSSHPublicKeyFingerprint string `json:"seeded_ssh_public_key_fingerprint,omitempty"` Docker bool `json:"docker"` @@ -152,7 +140,7 @@ type VMSetRequest struct { type ImageBuildRequest struct { Name string - BaseRootfs string + FromImage string Size string KernelPath string InitrdPath string diff --git a/internal/namegen/namegen.go b/internal/namegen/namegen.go new file mode 100644 index 0000000..b3edfad --- /dev/null +++ b/internal/namegen/namegen.go @@ -0,0 +1,71 @@ +package namegen + +import ( + "crypto/rand" + "encoding/binary" +) + +var adjectives = []string{ + "ace", "apt", "fit", "fun", "odd", "top", "able", "beau", "bold", "calm", + "chic", "cool", "deep", "deft", "easy", "epic", "fair", "fine", "free", "full", + "game", "glad", "glow", "good", "holy", "keen", "kind", "lean", "mild", "neat", + "nice", "open", "pure", "real", "snug", "spry", "tidy", "true", "warm", "wavy", + "wise", "adept", "agile", "alert", "alive", "ample", "angel", "awake", "aware", "brave", + "brisk", "chill", "clean", "clear", "close", "comic", "eager", "elite", "first", "fleet", + "fresh", "grace", "grand", "great", "happy", "hardy", "ideal", "jolly", "light", "lithe", + "loyal", "lucid", "lucky", "lunar", "magic", "merry", "nifty", "noble", "peppy", "perky", + "proud", "quick", "quiet", "ready", "regal", "savvy", "sharp", "smart", "solid", "sound", + "sunny", "super", "sweet", "swift", "vivid", "witty", "zesty", +} + +var substantives = []string{ + "ox", "aim", "air", "arm", "bud", "day", "hay", "jam", "jay", "joy", + "key", "map", "may", "nod", "ore", "pen", "sky", "sun", "way", "zen", + "ant", "ape", "auk", "bat", "bee", "cat", "cod", "cow", "dog", "elk", + "fox", "hen", "owl", "pig", "ram", "rat", "yak", "boar", "buck", "bull", + "calf", "carp", "crab", "crow", "deer", "dove", "fish", "foal", "frog", "goat", + "gull", "hare", "hawk", "ibex", "kiwi", "kudu", "lamb", "lion", "lynx", "mink", + "mole", "mule", "newt", "orca", "oryx", "puma", "seal", "slug", "stag", "swan", + "tern", "toad", "tuna", "wasp", "wolf", "zebu", "bison", "camel", "crane", "eagle", + "finch", "goose", "heron", "hippo", "horse", "hyena", "koala", "llama", "macaw", "moose", + "otter", "quail", "raven", "robin", "shark", "sheep", "shrew", "skunk", "sloth", "snail", + "squid", "tapir", "tiger", "trout", "whale", "zebra", "ally", "arch", "area", "aura", + "axis", "bank", "barn", "beam", "bell", "belt", "bend", "bird", "boat", "bond", + "book", "boot", "bowl", "brim", "calm", "camp", "card", "care", "cell", "city", + "clan", "club", "code", "core", "crux", "dawn", "deal", "film", "firm", "flag", + "flow", "foam", "gate", "gift", "glow", "hall", "hand", "harp", "hill", "home", + "hope", "host", "idea", "isle", "item", "keel", "knot", "land", "leaf", "link", + "lion", "loom", "love", "luck", "mark", "moon", "moss", "nook", "note", "pact", + "page", "path", "peak", "poem", "port", "ring", "road", "rock", "roof", "rule", + "sail", "seal", "seed", "song", "star", "tide", "tree", "tune", "walk", "ward", + "wave", "well", "wind", "wing", "wish", "wood", "work", "zone", "amity", "asset", + "bloom", "brook", "bunch", "charm", "chart", "cheer", "chord", "cliff", "cloud", "coast", + "comet", "craft", "crane", "crest", "crowd", "crown", "cycle", "faith", "field", "flame", + "fleet", "focus", "forge", "frame", "fruit", "glade", "grace", "grain", "grove", "guide", + "guild", "haven", "heart", "honey", "honor", "humor", "image", "index", "jewel", "judge", + "kudos", "lumen", "lunar", "magic", "march", "marsh", "mercy", "model", "moral", "music", + "niche", "oasis", "ocean", "opera", "orbit", "order", "peace", "pearl", "petal", "phase", + "piano", "pilot", "place", "plaza", "prism", "proof", "pulse", "quest", "quiet", "quill", + "radar", "rally", "range", "realm", "reign", "river", "route", "scene", "scope", "score", + "shade", "shape", "shore", "skill", "spark", "spice", "spire", "spoke", "stone", "story", + "table", "token", "trend", "tribe", "trust", "unity", "valor", "value", "verse", "vista", + "voice", "world", +} + +func Generate() string { + if len(adjectives) == 0 || len(substantives) == 0 { + return "" + } + return adjectives[randomIndex(len(adjectives))] + "-" + substantives[randomIndex(len(substantives))] +} + +func randomIndex(length int) int { + if length <= 1 { + return 0 + } + var buf [8]byte + if _, err := rand.Read(buf[:]); err != nil { + return 0 + } + return int(binary.BigEndian.Uint64(buf[:]) % uint64(length)) +} diff --git a/internal/paths/paths.go b/internal/paths/paths.go index 8608a19..0eeacba 100644 --- a/internal/paths/paths.go +++ b/internal/paths/paths.go @@ -5,10 +5,7 @@ import ( "fmt" "os" "path/filepath" - "strconv" "strings" - - "banger/internal/runtimebundle" ) type Layout struct { @@ -69,71 +66,6 @@ func Ensure(layout Layout) error { var executablePath = os.Executable -func ResolveRuntimeDir(configuredRuntimeDir, deprecatedRepoRoot string) string { - for _, candidate := range []string{ - os.Getenv("BANGER_RUNTIME_DIR"), - os.Getenv("BANGER_REPO_ROOT"), - configuredRuntimeDir, - deprecatedRepoRoot, - } { - if candidate = strings.TrimSpace(candidate); candidate != "" { - return filepath.Clean(candidate) - } - } - exe, err := executablePath() - if err != nil { - return "" - } - exeDir := filepath.Dir(exe) - if filepath.Base(exeDir) == "bin" { - if filepath.Base(filepath.Dir(exeDir)) == "build" { - buildRuntimeDir := filepath.Clean(filepath.Join(exeDir, "..", "runtime")) - if HasRuntimeBundle(buildRuntimeDir) { - return buildRuntimeDir - } - } - installRuntimeDir := filepath.Clean(filepath.Join(exeDir, "..", "lib", "banger")) - if HasRuntimeBundle(installRuntimeDir) { - return installRuntimeDir - } - } - for _, sourceRuntimeDir := range []string{ - filepath.Join(exeDir, "build", "runtime"), - filepath.Join(exeDir, "runtime"), - } { - if HasRuntimeBundle(sourceRuntimeDir) { - return sourceRuntimeDir - } - } - return "" -} - -func HasRuntimeBundle(dir string) bool { - if strings.TrimSpace(dir) == "" { - return false - } - if _, err := runtimebundle.LoadBundleMetadata(dir); err == nil { - return true - } - required := []string{ - "firecracker", - "customize.sh", - "packages.apt", - "wtf/root/boot/vmlinux-6.8.0-94-generic", - } - for _, name := range required { - if _, err := os.Stat(filepath.Join(dir, name)); err != nil { - return false - } - } - for _, name := range []string{"rootfs-docker.ext4", "rootfs.ext4"} { - if _, err := os.Stat(filepath.Join(dir, name)); err == nil { - return true - } - } - return false -} - func BangerdPath() (string, error) { if env := os.Getenv("BANGER_DAEMON_BIN"); env != "" { return env, nil @@ -154,8 +86,33 @@ func BangerdPath() (string, error) { return "", errors.New("bangerd binary not found next to banger; run `make build`") } -func RuntimeBundleHint() string { - return "run `make runtime-bundle` or set runtime_dir in ~/.config/banger/config.toml" +func CompanionBinaryPath(name string) (string, error) { + envNames := []string{ + "BANGER_" + strings.ToUpper(strings.NewReplacer("-", "_", ".", "_").Replace(name)) + "_BIN", + } + if trimmed, ok := strings.CutPrefix(name, "banger-"); ok { + envNames = append(envNames, "BANGER_"+strings.ToUpper(strings.NewReplacer("-", "_", ".", "_").Replace(trimmed))+"_BIN") + } + for _, envName := range envNames { + if env := strings.TrimSpace(os.Getenv(envName)); env != "" { + return env, nil + } + } + exe, err := executablePath() + if err != nil { + return "", err + } + exeDir := filepath.Dir(exe) + for _, candidate := range []string{ + filepath.Join(exeDir, name), + filepath.Join(exeDir, "..", "lib", "banger", name), + filepath.Join(exeDir, "..", "libexec", "banger", name), + } { + if _, err := os.Stat(candidate); err == nil { + return candidate, nil + } + } + return "", fmt.Errorf("%s companion binary not found; run `make build` or reinstall banger", name) } func getenvDefault(key, fallback string) string { @@ -164,7 +121,3 @@ func getenvDefault(key, fallback string) string { } return fallback } - -func RuntimeFallbackLabel() string { - return strconv.Itoa(os.Getuid()) -} diff --git a/internal/paths/paths_test.go b/internal/paths/paths_test.go index 68771a9..5ec7a53 100644 --- a/internal/paths/paths_test.go +++ b/internal/paths/paths_test.go @@ -1,44 +1,29 @@ package paths import ( - "encoding/json" "os" "path/filepath" "testing" - - "banger/internal/runtimebundle" ) -func TestResolveRuntimeDirPrefersEnv(t *testing.T) { - t.Setenv("BANGER_RUNTIME_DIR", "/env/runtime") +func TestCompanionBinaryPathPrefersEnv(t *testing.T) { + t.Setenv("BANGER_VSOCK_AGENT_BIN", "/tmp/custom-vsock-agent") - if got := ResolveRuntimeDir("/config/runtime", "/deprecated/repo"); got != "/env/runtime" { - t.Fatalf("ResolveRuntimeDir() = %q, want /env/runtime", got) + got, err := CompanionBinaryPath("banger-vsock-agent") + if err != nil { + t.Fatalf("CompanionBinaryPath: %v", err) + } + if got != "/tmp/custom-vsock-agent" { + t.Fatalf("CompanionBinaryPath() = %q", got) } } -func TestResolveRuntimeDirUsesInstalledLayout(t *testing.T) { +func TestCompanionBinaryPathUsesSiblingBinary(t *testing.T) { root := t.TempDir() - runtimeDir := filepath.Join(root, "lib", "banger") - createRuntimeBundle(t, runtimeDir) - - origExecutablePath := executablePath - executablePath = func() (string, error) { - return filepath.Join(root, "bin", "banger"), nil + companion := filepath.Join(root, "banger-vsock-agent") + if err := os.WriteFile(companion, []byte("test"), 0o755); err != nil { + t.Fatalf("write companion: %v", err) } - t.Cleanup(func() { - executablePath = origExecutablePath - }) - - if got := ResolveRuntimeDir("", ""); got != runtimeDir { - t.Fatalf("ResolveRuntimeDir() = %q, want %q", got, runtimeDir) - } -} - -func TestResolveRuntimeDirUsesBuildRuntimeForSourceCheckoutBinary(t *testing.T) { - root := t.TempDir() - runtimeDir := filepath.Join(root, "build", "runtime") - createRuntimeBundle(t, runtimeDir) origExecutablePath := executablePath executablePath = func() (string, error) { @@ -48,64 +33,38 @@ func TestResolveRuntimeDirUsesBuildRuntimeForSourceCheckoutBinary(t *testing.T) executablePath = origExecutablePath }) - if got := ResolveRuntimeDir("", ""); got != runtimeDir { - t.Fatalf("ResolveRuntimeDir() = %q, want %q", got, runtimeDir) + got, err := CompanionBinaryPath("banger-vsock-agent") + if err != nil { + t.Fatalf("CompanionBinaryPath: %v", err) + } + if got != companion { + t.Fatalf("CompanionBinaryPath() = %q, want %q", got, companion) } } -func TestResolveRuntimeDirUsesBuildRuntimeForBuildBinExecutable(t *testing.T) { +func TestCompanionBinaryPathUsesInstalledLibDir(t *testing.T) { root := t.TempDir() - runtimeDir := filepath.Join(root, "build", "runtime") - createRuntimeBundle(t, runtimeDir) + companion := filepath.Join(root, "lib", "banger", "banger-vsock-agent") + if err := os.MkdirAll(filepath.Dir(companion), 0o755); err != nil { + t.Fatalf("mkdir companion dir: %v", err) + } + if err := os.WriteFile(companion, []byte("test"), 0o755); err != nil { + t.Fatalf("write companion: %v", err) + } origExecutablePath := executablePath executablePath = func() (string, error) { - return filepath.Join(root, "build", "bin", "banger"), nil + return filepath.Join(root, "bin", "banger"), nil } t.Cleanup(func() { executablePath = origExecutablePath }) - if got := ResolveRuntimeDir("", ""); got != runtimeDir { - t.Fatalf("ResolveRuntimeDir() = %q, want %q", got, runtimeDir) - } -} - -func createRuntimeBundle(t *testing.T, runtimeDir string) { - t.Helper() - metadata := runtimebundle.BundleMetadata{ - FirecrackerBin: "bin/firecracker", - SSHKeyPath: "keys/id_ed25519", - NamegenPath: "bin/namegen", - CustomizeScript: "scripts/customize.sh", - VSockAgentPath: "bin/banger-vsock-agent", - DefaultPackages: "config/packages.apt", - DefaultRootfs: "images/rootfs-docker.ext4", - DefaultKernel: "kernels/vmlinux", - } - for _, rel := range []string{ - metadata.FirecrackerBin, - metadata.SSHKeyPath, - metadata.NamegenPath, - metadata.CustomizeScript, - metadata.VSockAgentPath, - metadata.DefaultPackages, - metadata.DefaultRootfs, - metadata.DefaultKernel, - } { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("mkdir %s: %v", filepath.Dir(path), err) - } - if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { - t.Fatalf("write %s: %v", path, err) - } - } - data, err := json.Marshal(metadata) + got, err := CompanionBinaryPath("banger-vsock-agent") if err != nil { - t.Fatalf("Marshal: %v", err) + t.Fatalf("CompanionBinaryPath: %v", err) } - if err := os.WriteFile(filepath.Join(runtimeDir, runtimebundle.BundleMetadataFile), data, 0o644); err != nil { - t.Fatalf("write bundle metadata: %v", err) + if got != companion { + t.Fatalf("CompanionBinaryPath() = %q, want %q", got, companion) } } diff --git a/internal/runtimebundle/bundle.go b/internal/runtimebundle/bundle.go deleted file mode 100644 index 111cb60..0000000 --- a/internal/runtimebundle/bundle.go +++ /dev/null @@ -1,497 +0,0 @@ -package runtimebundle - -import ( - "archive/tar" - "compress/gzip" - "context" - "crypto/sha256" - "encoding/hex" - "encoding/json" - "errors" - "fmt" - "io" - "net/http" - "net/url" - "os" - "path/filepath" - "sort" - "strings" - - toml "github.com/pelletier/go-toml" -) - -type Manifest struct { - Version string `toml:"version"` - URL string `toml:"url"` - SHA256 string `toml:"sha256"` - BundleRoot string `toml:"bundle_root"` - RequiredPaths []string `toml:"required_paths"` - BundleMeta BundleMetadata `toml:"bundle_metadata"` -} - -type BundleMetadata struct { - FirecrackerBin string `json:"firecracker_bin" toml:"firecracker_bin"` - SSHKeyPath string `json:"ssh_key_path" toml:"ssh_key_path"` - NamegenPath string `json:"namegen_path" toml:"namegen_path"` - CustomizeScript string `json:"customize_script" toml:"customize_script"` - VSockAgentPath string `json:"vsock_agent_path,omitempty" toml:"vsock_agent_path"` - VSockPingHelperPath string `json:"vsock_ping_helper_path,omitempty" toml:"vsock_ping_helper_path"` - DefaultPackages string `json:"default_packages_file" toml:"default_packages_file"` - DefaultRootfs string `json:"default_rootfs" toml:"default_rootfs"` - DefaultBaseRootfs string `json:"default_base_rootfs,omitempty" toml:"default_base_rootfs"` - DefaultWorkSeed string `json:"default_work_seed,omitempty" toml:"default_work_seed"` - DefaultKernel string `json:"default_kernel" toml:"default_kernel"` - DefaultInitrd string `json:"default_initrd,omitempty" toml:"default_initrd"` - DefaultModulesDir string `json:"default_modules_dir,omitempty" toml:"default_modules_dir"` -} - -const BundleMetadataFile = "bundle.json" - -func LoadManifest(path string) (Manifest, error) { - data, err := os.ReadFile(path) - if err != nil { - return Manifest{}, err - } - var manifest Manifest - if err := toml.Unmarshal(data, &manifest); err != nil { - return Manifest{}, err - } - manifest.BundleRoot = strings.TrimSpace(manifest.BundleRoot) - manifest.URL = strings.TrimSpace(manifest.URL) - manifest.SHA256 = strings.ToLower(strings.TrimSpace(manifest.SHA256)) - manifest.BundleMeta = normalizeBundleMetadata(manifest.BundleMeta) - for i, required := range manifest.RequiredPaths { - manifest.RequiredPaths[i] = filepath.Clean(strings.TrimSpace(required)) - } - sort.Strings(manifest.RequiredPaths) - if len(manifest.RequiredPaths) == 0 { - return Manifest{}, fmt.Errorf("runtime bundle manifest %s has no required_paths", path) - } - return manifest, nil -} - -func Bootstrap(ctx context.Context, manifest Manifest, manifestPath, outDir string) error { - if manifest.URL == "" { - return fmt.Errorf("runtime bundle manifest %s has no url; point a local manifest copy at a staged or published runtime bundle archive", manifestPath) - } - if manifest.SHA256 == "" { - return fmt.Errorf("runtime bundle manifest %s has no sha256; add the checksum for the staged or published runtime bundle archive", manifestPath) - } - manifestDir := filepath.Dir(manifestPath) - parentDir := filepath.Dir(outDir) - if err := os.MkdirAll(parentDir, 0o755); err != nil { - return err - } - - workDir, err := os.MkdirTemp(parentDir, ".runtime-bundle-*") - if err != nil { - return err - } - defer os.RemoveAll(workDir) - - archivePath := filepath.Join(workDir, "bundle.tar.gz") - if err := downloadArchive(ctx, resolveSource(manifestDir, manifest.URL), archivePath); err != nil { - return err - } - sum, err := fileSHA256(archivePath) - if err != nil { - return err - } - if sum != manifest.SHA256 { - return fmt.Errorf("runtime bundle checksum mismatch: got %s want %s", sum, manifest.SHA256) - } - - extractDir := filepath.Join(workDir, "extract") - if err := extractTarGz(archivePath, extractDir); err != nil { - return err - } - - bundleDir := extractDir - if manifest.BundleRoot != "" { - bundleDir = filepath.Join(extractDir, manifest.BundleRoot) - } - if err := ValidateBundle(bundleDir, manifest.RequiredPaths); err != nil { - return err - } - if _, err := LoadBundleMetadata(bundleDir); err != nil && !errors.Is(err, os.ErrNotExist) { - return err - } - - stageDir := filepath.Join(workDir, "stage") - if err := os.Rename(bundleDir, stageDir); err != nil { - return err - } - - if err := os.RemoveAll(outDir); err != nil { - return err - } - if err := os.Rename(stageDir, outDir); err != nil { - return err - } - return nil -} - -func ValidateBundle(bundleDir string, requiredPaths []string) error { - for _, rel := range requiredPaths { - if rel == "." || strings.HasPrefix(rel, "..") { - return fmt.Errorf("invalid required bundle path: %s", rel) - } - if _, err := os.Stat(filepath.Join(bundleDir, rel)); err != nil { - return fmt.Errorf("runtime bundle missing %s", rel) - } - } - return nil -} - -func Package(runtimeDir, outArchive string, manifest Manifest) (string, error) { - if err := ValidateBundle(runtimeDir, manifest.RequiredPaths); err != nil { - return "", err - } - metadata, err := metadataArchiveBytes(runtimeDir, manifest.BundleMeta) - if err != nil { - return "", err - } - if err := os.MkdirAll(filepath.Dir(outArchive), 0o755); err != nil { - return "", err - } - file, err := os.Create(outArchive) - if err != nil { - return "", err - } - defer file.Close() - - hash := sha256.New() - multi := io.MultiWriter(file, hash) - gz := gzip.NewWriter(multi) - defer gz.Close() - tw := tar.NewWriter(gz) - defer tw.Close() - - for _, rel := range manifest.RequiredPaths { - if err := addPathToArchive(tw, runtimeDir, manifest.BundleRoot, rel); err != nil { - return "", err - } - } - if len(metadata) != 0 { - if err := addBytesToArchive(tw, manifest.BundleRoot, BundleMetadataFile, metadata, 0o644); err != nil { - return "", err - } - } - if err := tw.Close(); err != nil { - return "", err - } - if err := gz.Close(); err != nil { - return "", err - } - return hex.EncodeToString(hash.Sum(nil)), nil -} - -func LoadBundleMetadata(runtimeDir string) (BundleMetadata, error) { - path := filepath.Join(runtimeDir, BundleMetadataFile) - data, err := os.ReadFile(path) - if err != nil { - return BundleMetadata{}, err - } - var meta BundleMetadata - if err := json.Unmarshal(data, &meta); err != nil { - return BundleMetadata{}, fmt.Errorf("parse %s: %w", path, err) - } - meta = normalizeBundleMetadata(meta) - if err := validateBundleMetadata(runtimeDir, meta); err != nil { - return BundleMetadata{}, err - } - return meta, nil -} - -func validateBundleMetadata(runtimeDir string, meta BundleMetadata) error { - required := []struct { - value string - label string - }{ - {meta.FirecrackerBin, "firecracker_bin"}, - {meta.SSHKeyPath, "ssh_key_path"}, - {meta.NamegenPath, "namegen_path"}, - {meta.CustomizeScript, "customize_script"}, - {meta.VSockAgentPath, "vsock_agent_path"}, - {meta.DefaultPackages, "default_packages_file"}, - {meta.DefaultRootfs, "default_rootfs"}, - {meta.DefaultKernel, "default_kernel"}, - } - for _, field := range required { - if strings.TrimSpace(field.value) == "" { - return fmt.Errorf("runtime bundle metadata missing %s", field.label) - } - } - for _, field := range []struct { - value string - label string - required bool - }{ - {meta.FirecrackerBin, "firecracker_bin", true}, - {meta.SSHKeyPath, "ssh_key_path", true}, - {meta.NamegenPath, "namegen_path", true}, - {meta.CustomizeScript, "customize_script", true}, - {meta.VSockAgentPath, "vsock_agent_path", true}, - {meta.DefaultPackages, "default_packages_file", true}, - {meta.DefaultRootfs, "default_rootfs", true}, - {meta.DefaultBaseRootfs, "default_base_rootfs", false}, - {meta.DefaultWorkSeed, "default_work_seed", false}, - {meta.DefaultKernel, "default_kernel", true}, - {meta.DefaultInitrd, "default_initrd", false}, - {meta.DefaultModulesDir, "default_modules_dir", false}, - } { - if strings.TrimSpace(field.value) == "" { - continue - } - resolved, err := resolveMetadataPath(runtimeDir, field.value) - if err != nil { - return fmt.Errorf("runtime bundle metadata %s: %w", field.label, err) - } - if _, err := os.Stat(resolved); err != nil { - if field.required || !errors.Is(err, os.ErrNotExist) { - return fmt.Errorf("runtime bundle metadata %s points to missing path %s", field.label, resolved) - } - } - } - return nil -} - -func resolveMetadataPath(runtimeDir, rel string) (string, error) { - rel = filepath.Clean(strings.TrimSpace(rel)) - if rel == "." || rel == "" || filepath.IsAbs(rel) || strings.HasPrefix(rel, "..") { - return "", fmt.Errorf("invalid relative path %q", rel) - } - return filepath.Join(runtimeDir, rel), nil -} - -func metadataArchiveBytes(runtimeDir string, meta BundleMetadata) ([]byte, error) { - meta = normalizeBundleMetadata(meta) - if strings.TrimSpace(meta.FirecrackerBin) == "" && - strings.TrimSpace(meta.SSHKeyPath) == "" && - strings.TrimSpace(meta.NamegenPath) == "" && - strings.TrimSpace(meta.CustomizeScript) == "" && - strings.TrimSpace(meta.VSockAgentPath) == "" && - strings.TrimSpace(meta.DefaultPackages) == "" && - strings.TrimSpace(meta.DefaultRootfs) == "" && - strings.TrimSpace(meta.DefaultBaseRootfs) == "" && - strings.TrimSpace(meta.DefaultWorkSeed) == "" && - strings.TrimSpace(meta.DefaultKernel) == "" && - strings.TrimSpace(meta.DefaultInitrd) == "" && - strings.TrimSpace(meta.DefaultModulesDir) == "" { - return nil, nil - } - if err := validateBundleMetadata(runtimeDir, meta); err != nil { - return nil, err - } - return json.MarshalIndent(meta, "", " ") -} - -func normalizeBundleMetadata(meta BundleMetadata) BundleMetadata { - meta.FirecrackerBin = strings.TrimSpace(meta.FirecrackerBin) - meta.SSHKeyPath = strings.TrimSpace(meta.SSHKeyPath) - meta.NamegenPath = strings.TrimSpace(meta.NamegenPath) - meta.CustomizeScript = strings.TrimSpace(meta.CustomizeScript) - meta.VSockAgentPath = strings.TrimSpace(meta.VSockAgentPath) - meta.VSockPingHelperPath = strings.TrimSpace(meta.VSockPingHelperPath) - if meta.VSockAgentPath == "" { - meta.VSockAgentPath = meta.VSockPingHelperPath - } - meta.DefaultPackages = strings.TrimSpace(meta.DefaultPackages) - meta.DefaultRootfs = strings.TrimSpace(meta.DefaultRootfs) - meta.DefaultBaseRootfs = strings.TrimSpace(meta.DefaultBaseRootfs) - meta.DefaultWorkSeed = strings.TrimSpace(meta.DefaultWorkSeed) - meta.DefaultKernel = strings.TrimSpace(meta.DefaultKernel) - meta.DefaultInitrd = strings.TrimSpace(meta.DefaultInitrd) - meta.DefaultModulesDir = strings.TrimSpace(meta.DefaultModulesDir) - return meta -} - -func addPathToArchive(tw *tar.Writer, runtimeDir, bundleRoot, rel string) error { - srcPath := filepath.Join(runtimeDir, rel) - info, err := os.Lstat(srcPath) - if err != nil { - return err - } - archiveName := rel - if bundleRoot != "" { - archiveName = filepath.Join(bundleRoot, rel) - } - if info.IsDir() { - header, err := tar.FileInfoHeader(info, "") - if err != nil { - return err - } - header.Name = filepath.ToSlash(archiveName) + "/" - if err := tw.WriteHeader(header); err != nil { - return err - } - entries, err := os.ReadDir(srcPath) - if err != nil { - return err - } - for _, entry := range entries { - childRel := filepath.Join(rel, entry.Name()) - if err := addPathToArchive(tw, runtimeDir, bundleRoot, childRel); err != nil { - return err - } - } - return nil - } - - header, err := tar.FileInfoHeader(info, "") - if err != nil { - return err - } - header.Name = filepath.ToSlash(archiveName) - if err := tw.WriteHeader(header); err != nil { - return err - } - file, err := os.Open(srcPath) - if err != nil { - return err - } - defer file.Close() - _, err = io.Copy(tw, file) - return err -} - -func addBytesToArchive(tw *tar.Writer, bundleRoot, rel string, data []byte, mode int64) error { - name := rel - if bundleRoot != "" { - name = filepath.Join(bundleRoot, rel) - } - header := &tar.Header{ - Name: filepath.ToSlash(name), - Mode: mode, - Size: int64(len(data)), - } - if err := tw.WriteHeader(header); err != nil { - return err - } - _, err := tw.Write(data) - return err -} - -func resolveSource(manifestDir, source string) string { - parsed, err := url.Parse(source) - if err == nil && parsed.Scheme != "" { - return source - } - if filepath.IsAbs(source) { - return source - } - return filepath.Join(manifestDir, source) -} - -func downloadArchive(ctx context.Context, source, dst string) error { - switch { - case strings.HasPrefix(source, "http://"), strings.HasPrefix(source, "https://"): - req, err := http.NewRequestWithContext(ctx, http.MethodGet, source, nil) - if err != nil { - return err - } - resp, err := http.DefaultClient.Do(req) - if err != nil { - return err - } - defer resp.Body.Close() - if resp.StatusCode != http.StatusOK { - return fmt.Errorf("download runtime bundle: %s", resp.Status) - } - return writeFileFromReader(dst, resp.Body) - case strings.HasPrefix(source, "file://"): - parsed, err := url.Parse(source) - if err != nil { - return err - } - return copyFile(parsed.Path, dst) - default: - return copyFile(source, dst) - } -} - -func writeFileFromReader(dst string, reader io.Reader) error { - file, err := os.Create(dst) - if err != nil { - return err - } - defer file.Close() - _, err = io.Copy(file, reader) - return err -} - -func copyFile(src, dst string) error { - in, err := os.Open(src) - if err != nil { - return err - } - defer in.Close() - return writeFileFromReader(dst, in) -} - -func extractTarGz(archivePath, outDir string) error { - if err := os.MkdirAll(outDir, 0o755); err != nil { - return err - } - file, err := os.Open(archivePath) - if err != nil { - return err - } - defer file.Close() - gz, err := gzip.NewReader(file) - if err != nil { - return err - } - defer gz.Close() - tr := tar.NewReader(gz) - for { - header, err := tr.Next() - if err == io.EOF { - return nil - } - if err != nil { - return err - } - name := filepath.Clean(header.Name) - if name == "." || strings.HasPrefix(name, "..") || filepath.IsAbs(name) { - return fmt.Errorf("invalid archive entry: %s", header.Name) - } - target := filepath.Join(outDir, name) - switch header.Typeflag { - case tar.TypeDir: - if err := os.MkdirAll(target, os.FileMode(header.Mode)); err != nil { - return err - } - case tar.TypeReg, tar.TypeRegA: - if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { - return err - } - file, err := os.OpenFile(target, os.O_CREATE|os.O_RDWR|os.O_TRUNC, os.FileMode(header.Mode)) - if err != nil { - return err - } - if _, err := io.Copy(file, tr); err != nil { - file.Close() - return err - } - if err := file.Close(); err != nil { - return err - } - default: - return fmt.Errorf("unsupported archive entry type: %s", header.Name) - } - } -} - -func fileSHA256(path string) (string, error) { - file, err := os.Open(path) - if err != nil { - return "", err - } - defer file.Close() - hash := sha256.New() - if _, err := io.Copy(hash, file); err != nil { - return "", err - } - return hex.EncodeToString(hash.Sum(nil)), nil -} diff --git a/internal/runtimebundle/bundle_test.go b/internal/runtimebundle/bundle_test.go deleted file mode 100644 index ea8c56c..0000000 --- a/internal/runtimebundle/bundle_test.go +++ /dev/null @@ -1,288 +0,0 @@ -package runtimebundle - -import ( - "archive/tar" - "bytes" - "compress/gzip" - "context" - "crypto/sha256" - "encoding/hex" - "encoding/json" - "os" - "path/filepath" - "strings" - "testing" -) - -func TestBootstrapExtractsBundleAndValidatesChecksum(t *testing.T) { - manifestDir := t.TempDir() - bundleData := buildArchive(t, map[string]string{ - "runtime/firecracker": "fc", - "runtime/id_ed25519": "key", - "runtime/namegen": "namegen", - "runtime/banger-vsock-agent": "agent", - "runtime/customize.sh": "#!/bin/bash\n", - "runtime/packages.sh": "#!/bin/bash\n", - "runtime/packages.apt": "vim\n", - "runtime/rootfs-docker.ext4": "rootfs", - "runtime/wtf/root/boot/vmlinux-6.8.0-94-generic": "kernel", - "runtime/wtf/root/boot/initrd.img-6.8.0-94-generic": "initrd", - "runtime/wtf/root/lib/modules/6.8.0-94-generic/modules.dep": "dep", - "runtime/bundle.json": mustJSON(t, BundleMetadata{FirecrackerBin: "firecracker", SSHKeyPath: "id_ed25519", NamegenPath: "namegen", CustomizeScript: "customize.sh", VSockAgentPath: "banger-vsock-agent", DefaultPackages: "packages.apt", DefaultRootfs: "rootfs-docker.ext4", DefaultKernel: "wtf/root/boot/vmlinux-6.8.0-94-generic", DefaultInitrd: "wtf/root/boot/initrd.img-6.8.0-94-generic", DefaultModulesDir: "wtf/root/lib/modules/6.8.0-94-generic"}), - }) - archivePath := filepath.Join(manifestDir, "bundle.tar.gz") - if err := os.WriteFile(archivePath, bundleData, 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - - manifest := Manifest{ - URL: "./bundle.tar.gz", - SHA256: sha256Hex(bundleData), - BundleRoot: "runtime", - RequiredPaths: []string{"firecracker", "banger-vsock-agent", "customize.sh", "packages.apt", "rootfs-docker.ext4", "wtf/root/boot/vmlinux-6.8.0-94-generic", "wtf/root/lib/modules/6.8.0-94-generic"}, - } - outDir := filepath.Join(t.TempDir(), "runtime") - if err := Bootstrap(context.Background(), manifest, filepath.Join(manifestDir, "runtime-bundle.toml"), outDir); err != nil { - t.Fatalf("Bootstrap: %v", err) - } - for _, rel := range manifest.RequiredPaths { - if _, err := os.Stat(filepath.Join(outDir, rel)); err != nil { - t.Fatalf("runtime missing %s: %v", rel, err) - } - } -} - -func TestBootstrapRejectsChecksumMismatch(t *testing.T) { - manifestDir := t.TempDir() - archivePath := filepath.Join(manifestDir, "bundle.tar.gz") - if err := os.WriteFile(archivePath, []byte("not-a-tarball"), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - manifest := Manifest{ - URL: "./bundle.tar.gz", - SHA256: strings.Repeat("0", 64), - BundleRoot: "runtime", - RequiredPaths: []string{"firecracker"}, - } - err := Bootstrap(context.Background(), manifest, filepath.Join(manifestDir, "runtime-bundle.toml"), filepath.Join(t.TempDir(), "runtime")) - if err == nil || !strings.Contains(err.Error(), "checksum mismatch") { - t.Fatalf("Bootstrap() error = %v, want checksum mismatch", err) - } -} - -func TestBootstrapRejectsMissingURLWithLocalManifestGuidance(t *testing.T) { - manifest := Manifest{ - SHA256: strings.Repeat("0", 64), - BundleRoot: "runtime", - RequiredPaths: []string{"firecracker"}, - } - err := Bootstrap(context.Background(), manifest, filepath.Join(t.TempDir(), "runtime-bundle.toml"), filepath.Join(t.TempDir(), "runtime")) - if err == nil || !strings.Contains(err.Error(), "local manifest copy") { - t.Fatalf("Bootstrap() error = %v, want local manifest guidance", err) - } -} - -func TestBootstrapRejectsMissingSHAWithArchiveGuidance(t *testing.T) { - manifest := Manifest{ - URL: "./bundle.tar.gz", - BundleRoot: "runtime", - RequiredPaths: []string{"firecracker"}, - } - err := Bootstrap(context.Background(), manifest, filepath.Join(t.TempDir(), "runtime-bundle.toml"), filepath.Join(t.TempDir(), "runtime")) - if err == nil || !strings.Contains(err.Error(), "staged or published runtime bundle archive") { - t.Fatalf("Bootstrap() error = %v, want archive guidance", err) - } -} - -func TestPackageWritesArchive(t *testing.T) { - runtimeDir := t.TempDir() - for _, rel := range []string{ - "firecracker", - "id_ed25519", - "namegen", - "banger-vsock-agent", - "customize.sh", - "packages.apt", - "rootfs-docker.ext4", - "wtf/root/boot/vmlinux-6.8.0-94-generic", - "wtf/root/boot/initrd.img-6.8.0-94-generic", - "wtf/root/lib/modules/6.8.0-94-generic", - } { - path := filepath.Join(runtimeDir, rel) - if rel == "wtf/root/lib/modules/6.8.0-94-generic" { - if err := os.MkdirAll(path, 0o755); err != nil { - t.Fatalf("MkdirAll: %v", err) - } - if err := os.WriteFile(filepath.Join(path, "modules.dep"), []byte(rel), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - continue - } - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("MkdirAll: %v", err) - } - if err := os.WriteFile(path, []byte(rel), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - } - manifest := Manifest{ - BundleRoot: "runtime", - BundleMeta: BundleMetadata{ - FirecrackerBin: "firecracker", - SSHKeyPath: "id_ed25519", - NamegenPath: "namegen", - CustomizeScript: "customize.sh", - VSockAgentPath: "banger-vsock-agent", - DefaultPackages: "packages.apt", - DefaultRootfs: "rootfs-docker.ext4", - DefaultKernel: "wtf/root/boot/vmlinux-6.8.0-94-generic", - DefaultInitrd: "wtf/root/boot/initrd.img-6.8.0-94-generic", - DefaultModulesDir: "wtf/root/lib/modules/6.8.0-94-generic", - }, - RequiredPaths: []string{ - "firecracker", - "id_ed25519", - "namegen", - "banger-vsock-agent", - "customize.sh", - "packages.apt", - "rootfs-docker.ext4", - "wtf/root/boot/vmlinux-6.8.0-94-generic", - "wtf/root/boot/initrd.img-6.8.0-94-generic", - "wtf/root/lib/modules/6.8.0-94-generic", - }, - } - outArchive := filepath.Join(t.TempDir(), "bundle.tar.gz") - sum, err := Package(runtimeDir, outArchive, manifest) - if err != nil { - t.Fatalf("Package: %v", err) - } - if sum == "" { - t.Fatalf("Package() returned empty checksum") - } - if _, err := os.Stat(outArchive); err != nil { - t.Fatalf("archive missing: %v", err) - } - runtimeOut := filepath.Join(t.TempDir(), "runtime") - if err := Bootstrap(context.Background(), Manifest{ - URL: outArchive, - SHA256: sum, - BundleRoot: "runtime", - RequiredPaths: manifest.RequiredPaths, - }, filepath.Join(t.TempDir(), "runtime-bundle.toml"), runtimeOut); err != nil { - t.Fatalf("Bootstrap packaged archive: %v", err) - } - if _, err := os.Stat(filepath.Join(runtimeOut, BundleMetadataFile)); err != nil { - t.Fatalf("bundle metadata missing after bootstrap: %v", err) - } - meta, err := LoadBundleMetadata(runtimeOut) - if err != nil { - t.Fatalf("LoadBundleMetadata: %v", err) - } - if meta.DefaultRootfs != manifest.BundleMeta.DefaultRootfs { - t.Fatalf("DefaultRootfs = %q, want %q", meta.DefaultRootfs, manifest.BundleMeta.DefaultRootfs) - } -} - -func TestLoadBundleMetadataRejectsMissingRequiredPath(t *testing.T) { - runtimeDir := t.TempDir() - for _, rel := range []string{"firecracker", "id_ed25519", "namegen", "banger-vsock-agent", "customize.sh", "packages.apt", "rootfs-docker.ext4"} { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("MkdirAll: %v", err) - } - if err := os.WriteFile(path, []byte(rel), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - } - data := mustJSON(t, BundleMetadata{ - FirecrackerBin: "firecracker", - SSHKeyPath: "id_ed25519", - NamegenPath: "namegen", - CustomizeScript: "customize.sh", - VSockAgentPath: "banger-vsock-agent", - DefaultPackages: "packages.apt", - DefaultRootfs: "rootfs-docker.ext4", - DefaultKernel: "missing-kernel", - }) - if err := os.WriteFile(filepath.Join(runtimeDir, BundleMetadataFile), []byte(data), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - if _, err := LoadBundleMetadata(runtimeDir); err == nil || !strings.Contains(err.Error(), "default_kernel") { - t.Fatalf("LoadBundleMetadata() error = %v, want default_kernel failure", err) - } -} - -func TestLoadBundleMetadataAcceptsLegacyVsockPingHelperPath(t *testing.T) { - runtimeDir := t.TempDir() - for _, rel := range []string{"firecracker", "id_ed25519", "namegen", "banger-vsock-pingd", "customize.sh", "packages.apt", "rootfs-docker.ext4", "wtf/root/boot/vmlinux-6.8.0-94-generic"} { - path := filepath.Join(runtimeDir, rel) - if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { - t.Fatalf("MkdirAll: %v", err) - } - if err := os.WriteFile(path, []byte(rel), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - } - data := mustJSON(t, BundleMetadata{ - FirecrackerBin: "firecracker", - SSHKeyPath: "id_ed25519", - NamegenPath: "namegen", - CustomizeScript: "customize.sh", - VSockPingHelperPath: "banger-vsock-pingd", - DefaultPackages: "packages.apt", - DefaultRootfs: "rootfs-docker.ext4", - DefaultKernel: "wtf/root/boot/vmlinux-6.8.0-94-generic", - }) - if err := os.WriteFile(filepath.Join(runtimeDir, BundleMetadataFile), []byte(data), 0o644); err != nil { - t.Fatalf("WriteFile: %v", err) - } - meta, err := LoadBundleMetadata(runtimeDir) - if err != nil { - t.Fatalf("LoadBundleMetadata: %v", err) - } - if meta.VSockAgentPath != "banger-vsock-pingd" { - t.Fatalf("VSockAgentPath = %q", meta.VSockAgentPath) - } -} - -func buildArchive(t *testing.T, files map[string]string) []byte { - t.Helper() - var buf bytes.Buffer - gz := gzip.NewWriter(&buf) - tw := tar.NewWriter(gz) - for name, contents := range files { - header := &tar.Header{ - Name: name, - Mode: 0o644, - Size: int64(len(contents)), - } - if err := tw.WriteHeader(header); err != nil { - t.Fatalf("WriteHeader(%s): %v", name, err) - } - if _, err := tw.Write([]byte(contents)); err != nil { - t.Fatalf("Write(%s): %v", name, err) - } - } - if err := tw.Close(); err != nil { - t.Fatalf("Close tar: %v", err) - } - if err := gz.Close(); err != nil { - t.Fatalf("Close gzip: %v", err) - } - return buf.Bytes() -} - -func sha256Hex(data []byte) string { - sum := sha256.Sum256(data) - return hex.EncodeToString(sum[:]) -} - -func mustJSON(t *testing.T, value any) string { - t.Helper() - data, err := json.Marshal(value) - if err != nil { - t.Fatalf("Marshal: %v", err) - } - return string(data) -} diff --git a/internal/store/store.go b/internal/store/store.go index f15ebfc..1ef1dca 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -120,8 +120,8 @@ func (s *Store) UpsertImage(ctx context.Context, image model.Image) error { const query = ` INSERT INTO images ( id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, - modules_dir, packages_path, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + modules_dir, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(id) DO UPDATE SET name=excluded.name, managed=excluded.managed, @@ -131,7 +131,6 @@ func (s *Store) UpsertImage(ctx context.Context, image model.Image) error { kernel_path=excluded.kernel_path, initrd_path=excluded.initrd_path, modules_dir=excluded.modules_dir, - packages_path=excluded.packages_path, build_size=excluded.build_size, seeded_ssh_public_key_fingerprint=excluded.seeded_ssh_public_key_fingerprint, docker=excluded.docker, @@ -146,7 +145,6 @@ func (s *Store) UpsertImage(ctx context.Context, image model.Image) error { image.KernelPath, image.InitrdPath, image.ModulesDir, - image.PackagesPath, image.BuildSize, image.SeededSSHPublicKeyFingerprint, boolToInt(image.Docker), @@ -157,15 +155,15 @@ func (s *Store) UpsertImage(ctx context.Context, image model.Image) error { } func (s *Store) GetImageByName(ctx context.Context, name string) (model.Image, error) { - return s.getImage(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, packages_path, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images WHERE name = ?", name) + return s.getImage(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images WHERE name = ?", name) } func (s *Store) GetImageByID(ctx context.Context, id string) (model.Image, error) { - return s.getImage(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, packages_path, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images WHERE id = ?", id) + return s.getImage(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images WHERE id = ?", id) } func (s *Store) ListImages(ctx context.Context) ([]model.Image, error) { - rows, err := s.db.QueryContext(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, packages_path, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images ORDER BY created_at ASC") + rows, err := s.db.QueryContext(ctx, "SELECT id, name, managed, artifact_dir, rootfs_path, work_seed_path, kernel_path, initrd_path, modules_dir, build_size, seeded_ssh_public_key_fingerprint, docker, created_at, updated_at FROM images ORDER BY created_at ASC") if err != nil { return nil, err } @@ -355,7 +353,6 @@ func scanImageRow(row scanner) (model.Image, error) { &image.KernelPath, &image.InitrdPath, &image.ModulesDir, - &image.PackagesPath, &image.BuildSize, &seededSSHPublicKeyFingerprint, &docker, diff --git a/internal/store/store_test.go b/internal/store/store_test.go index 0e7ea2a..164ad4e 100644 --- a/internal/store/store_test.go +++ b/internal/store/store_test.go @@ -344,7 +344,6 @@ func sampleImage(name string) model.Image { KernelPath: "/kernels/" + name, InitrdPath: "/initrd/" + name, ModulesDir: "/modules/" + name, - PackagesPath: "/packages/" + name + ".apt", BuildSize: "8G", SeededSSHPublicKeyFingerprint: "seeded-fingerprint", Docker: true, diff --git a/internal/system/system.go b/internal/system/system.go index 753b532..fc81fd5 100644 --- a/internal/system/system.go +++ b/internal/system/system.go @@ -84,6 +84,10 @@ func RequireCommands(ctx context.Context, commands ...string) error { return nil } +func LookupExecutable(name string) (string, error) { + return exec.LookPath(name) +} + func WriteJSON(path string, value any) error { data, err := json.MarshalIndent(value, "", " ") if err != nil { diff --git a/internal/webui/server.go b/internal/webui/server.go index d87dccb..0199b41 100644 --- a/internal/webui/server.go +++ b/internal/webui/server.go @@ -86,7 +86,7 @@ type vmSetForm struct { type imageBuildForm struct { Name string - BaseRootfs string + FromImage string Size string KernelPath string InitrdPath string @@ -101,7 +101,6 @@ type imageRegisterForm struct { KernelPath string InitrdPath string ModulesDir string - PackagesPath string Docker bool } @@ -524,13 +523,7 @@ func (s *Server) handleImageList(w http.ResponseWriter, r *http.Request) error { } func (s *Server) handleImageBuildForm(w http.ResponseWriter, r *http.Request) error { - cfg := s.backend.Config() - return s.renderImageBuildPage(w, r, imageBuildForm{ - BaseRootfs: cfg.DefaultBaseRootfs, - KernelPath: cfg.DefaultKernel, - InitrdPath: cfg.DefaultInitrd, - ModulesDir: cfg.DefaultModulesDir, - }, "") + return s.renderImageBuildPage(w, r, imageBuildForm{}, "") } func (s *Server) renderImageBuildPage(w http.ResponseWriter, r *http.Request, form imageBuildForm, formErr string) error { @@ -566,12 +559,7 @@ func (s *Server) handleImageBuild(w http.ResponseWriter, r *http.Request) error } func (s *Server) handleImageRegisterForm(w http.ResponseWriter, r *http.Request) error { - cfg := s.backend.Config() - return s.renderImageRegisterPage(w, r, imageRegisterForm{ - KernelPath: cfg.DefaultKernel, - InitrdPath: cfg.DefaultInitrd, - ModulesDir: cfg.DefaultModulesDir, - }, "") + return s.renderImageRegisterPage(w, r, imageRegisterForm{}, "") } func (s *Server) renderImageRegisterPage(w http.ResponseWriter, r *http.Request, form imageRegisterForm, formErr string) error { @@ -808,9 +796,6 @@ func (s *Server) pickerRoots() []pickerRoot { if layout.StateDir != "" { roots = append(roots, pickerRoot{Label: "State", Path: layout.StateDir}) } - if runtimeDir := s.backend.Config().RuntimeDir; runtimeDir != "" { - roots = append(roots, pickerRoot{Label: "Runtime", Path: runtimeDir}) - } result := make([]pickerRoot, 0, len(roots)) for _, root := range roots { root.Path = filepath.Clean(root.Path) @@ -998,7 +983,7 @@ func (s *Server) parseImageBuildForm(r *http.Request) (imageBuildForm, api.Image } form := imageBuildForm{ Name: strings.TrimSpace(r.FormValue("name")), - BaseRootfs: strings.TrimSpace(r.FormValue("base_rootfs")), + FromImage: strings.TrimSpace(r.FormValue("from_image")), Size: strings.TrimSpace(r.FormValue("size")), KernelPath: strings.TrimSpace(r.FormValue("kernel_path")), InitrdPath: strings.TrimSpace(r.FormValue("initrd_path")), @@ -1007,7 +992,7 @@ func (s *Server) parseImageBuildForm(r *http.Request) (imageBuildForm, api.Image } params := api.ImageBuildParams{ Name: form.Name, - BaseRootfs: form.BaseRootfs, + FromImage: form.FromImage, Size: form.Size, KernelPath: form.KernelPath, InitrdPath: form.InitrdPath, @@ -1028,7 +1013,6 @@ func (s *Server) parseImageRegisterForm(r *http.Request) (imageRegisterForm, api KernelPath: strings.TrimSpace(r.FormValue("kernel_path")), InitrdPath: strings.TrimSpace(r.FormValue("initrd_path")), ModulesDir: strings.TrimSpace(r.FormValue("modules_dir")), - PackagesPath: strings.TrimSpace(r.FormValue("packages_path")), Docker: r.FormValue("docker") == "on", } params := api.ImageRegisterParams{ @@ -1038,7 +1022,6 @@ func (s *Server) parseImageRegisterForm(r *http.Request) (imageRegisterForm, api KernelPath: form.KernelPath, InitrdPath: form.InitrdPath, ModulesDir: form.ModulesDir, - PackagesPath: form.PackagesPath, Docker: form.Docker, } return form, params, nil diff --git a/internal/webui/templates/images.html b/internal/webui/templates/images.html index 0b4fe3b..f8e884b 100644 --- a/internal/webui/templates/images.html +++ b/internal/webui/templates/images.html @@ -33,20 +33,14 @@ {{end}} {{define "image_build_content"}} -

Build a managed image from a base rootfs, then redirect into the async build progress view.

+

Build a managed image from an existing registered image, then redirect into the async build progress view.

{{if .ErrorMessage}}
{{.ErrorMessage}}
{{end}}
{{template "csrf_field" .}} - + -