Four targeted fixes from a race-condition audit of the daemon package.
None change behaviour on the happy path; each closes a window where a
concurrent or interrupted RPC could strand state on the host.
- KernelDelete now holds the same per-name lock as KernelPull /
readOrAutoPullKernel. Without it, a delete racing a concurrent
pull could remove files mid-write or land between the pull's
manifest write and its first use.
- cleanupRuntime no longer early-returns on an inner waitForExit
failure; DM snapshot, capability, and tap teardown always run and
every error is folded into the returned errors.Join. EBUSY against
a still-alive firecracker is benign and surfaces in the joined
error rather than stranding kernel state across daemon restarts.
- Per-name image / kernel pull locks switch from *sync.Mutex to a
1-buffered chan struct{}. Acquire is a select on ctx.Done(), so a
peer waiting behind a pull whose RPC was cancelled can bail out
instead of blocking forever on a pull nobody is consuming.
- setVMHandles writes the per-VM scratch file before updating the
in-memory cache. A daemon crash between the two now leaves disk
ahead of memory (recoverable: reconcile re-seeds the cache from
the file on next start) rather than memory ahead of disk (lost
handles → stranded DM/loops/tap).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three concurrency bugs surfaced by `make smoke JOBS=4` that all stem
from `vm.create` paths assuming single-caller semantics:
1. **Kernel auto-pull manifest race.** Parallel `vm.create` calls that
each need to auto-pull the same kernel ref both run kernelcat.Fetch
in parallel against the same /var/lib/banger/kernels/<name>/. Fetch
writes manifest.json non-atomically (truncate + write); the peer
reads it back mid-write and trips
"parse manifest for X: unexpected end of JSON input".
Fix: per-name `sync.Mutex` map on `ImageService` (kernelPullLock).
`KernelPull` and `readOrAutoPullKernel` both acquire it and re-check
`kernelcat.ReadLocal` after the lock so a peer who finished while we
waited is treated as success — `readOrAutoPullKernel` does NOT call
`s.KernelPull` because that path errors with "already pulled" on a
peer-success, which would be wrong for auto-pull. Different kernels
stay parallel.
2. **Image auto-pull race.** Same shape as the kernel race but on the
image side: parallel `vm.create` calls both run pullFromBundle /
pullFromOCI for the missing image (each ~minutes of OCI fetch +
ext4 build). The publishImage atom under imageOpsMu only protects
the rename + UpsertImage commit, so the loser does all the work
only to fail at the recheck with "image already exists".
Fix: per-name `sync.Mutex` map on `ImageService` (imagePullLock).
`findOrAutoPullImage` acquires it, re-checks FindImage, and only
then calls PullImage. Loser short-circuits with the
freshly-published image instead of redoing minutes of work.
PullImage's own publishImage recheck stays as defense-in-depth
for callers that bypass the auto-pull path.
3. **Work-seed refresh race.** When the host's SSH key has rotated
since an image was last refreshed, `ensureAuthorizedKeyOnWorkDisk`
triggers `refreshManagedWorkSeedFingerprint`, which rewrote the
shared work-seed.ext4 in place via e2rm + e2cp. Peer `vm.create`
calls doing parallel `MaterializeWorkDisk` rdumps observed a torn
ext4 image — "Superblock checksum does not match superblock".
Fix: stage the rewrite on a sibling tmpfile (`<seed>.refresh.<pid>-<ns>.tmp`)
and atomic-rename. Concurrent readers either have the file open
(kernel keeps the pre-rename inode alive) or open after the rename
(see the new inode) — never observe a partial state. Two parallel
refreshes are idempotent (same daemon, same SSH key) so unique tmp
names are enough; whichever rename lands last wins, with identical
content. UpsertImage runs after the rename so the recorded
fingerprint always matches what's on disk.
Plus one smoke harness fix: reclassify `vm_prune` from `pure` to
`global`. `vm prune -f` removes ALL stopped VMs system-wide, not just
the ones the scenario created — so a parallel peer scenario that
happens to have its VM in `created`/`stopped` momentarily gets wiped.
Moving prune to the post-pool serial phase keeps it from racing with
in-flight scenarios.
After all four fixes, `make smoke JOBS=4` passes 21/21 in 174s
(serial baseline 141s; the small overhead is the buffered-output and
`wait -n` semaphore cost — well worth the parallelism for fast-iter
work on a 32-core box).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second phase of splitting the daemon god-struct. ImageService now owns
all image + kernel registry operations: register/promote/delete/pull
for images (bundle + OCI paths), the six kernel commands, and the
shared SSH-key/work-seed injection helpers. imageOpsMu (the
publication-window lock) lives on the service; so do the three OCI
pull test seams pullAndFlatten / finalizePulledRootfs / bundleFetch.
The four files images.go, images_pull.go, image_seed.go, kernels.go
flipped their receivers from *Daemon to *ImageService.
FindImage moved with the service. Daemon keeps a thin FindImage
forwarder so callers reading the dispatch code see the obvious
facade and tests that pre-date the split still compile.
flattenNestedWorkHome — called from image_seed.go, vm_authsync.go,
and vm_disk.go across future service boundaries — became a
package-level helper taking a CommandRunner explicitly. Daemon keeps
a deprecated forwarder for now; the other services will use the
package form.
Lazy-init helper imageSvc() on Daemon mirrors hostNet() from
Phase 1, so test literals like &Daemon{store: db, runner: r, ...}
that don't spell out an ImageService still get a working one.
Tests that override the image test seams (autopull_test,
concurrency_test, images_pull_test, images_pull_bundle_test) now
assign d.img = &ImageService{...seams...}; the two-statement pattern
matches what Phase 1 established for HostNetwork.
Dispatch in daemon.go is cleaner now: every image/kernel RPC handler
is a single-liner forwarding to d.imageSvc().*. Phase 5 will do the
same for VM lifecycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces the headline feature of the kernel catalog: pulling a kernel
bundle over HTTP without any local build step.
Catalog format (internal/kernelcat/catalog.go):
- Catalog { Version, Entries } + CatEntry { Name, Distro, Arch,
KernelVersion, TarballURL, TarballSHA256, SizeBytes, Description }.
- catalog.json is embedded via go:embed and ships with each banger
binary. It starts empty (Phase 5's CI pipeline will populate it).
- Lookup(name) returns the matching entry or os.ErrNotExist.
Fetch (internal/kernelcat/fetch.go):
- HTTP GET with streaming SHA256 over the response body.
- zstd-decode (github.com/klauspost/compress/zstd) -> tar extract into
<kernelsDir>/<name>/.
- Hardens against path-traversal tarball entries (members whose
normalised path escapes the target dir, and unsafe symlink
targets) and sha256-mismatch downloads; any failure removes the
partially-populated target dir.
- Regular files, directories, and safe symlinks are supported; other
tar types (hardlinks, devices, fifos) are silently skipped.
- After extraction, recomputes sha256 over the on-disk vmlinux and
writes the manifest with Source="pull:<url>".
Daemon methods (internal/daemon/kernels.go):
- KernelPull(ctx, {Name, Force}) - lookup in embedded catalog, refuse
overwrite unless Force, delegate to kernelcat.Fetch.
- KernelCatalog(ctx) - return the embedded catalog annotated per-entry
with whether it has been pulled locally.
RPC: kernel.pull, kernel.catalog dispatch cases.
CLI:
- `banger kernel pull <name> [--force]`.
- `banger kernel list --available` prints the catalog with a
pulled/available STATE column and a human-readable size.
Tests: fetch round-trip (extract + manifest + sha256), sha256 mismatch
rejection with cleanup, missing-vmlinux rejection, path-traversal
rejection, HTTP error propagation, catalog parsing, lookup,
pulled-status reconciliation. All 20 packages green.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`banger kernel import <name> --from <dir>` copies a staged kernel
bundle into the local catalog. <dir> is the output of
`make void-kernel` or `make alpine-kernel` (build/manual/void-kernel/
or build/manual/alpine-kernel/).
kernelcat.DiscoverPaths locates artifacts under <dir>:
1. Prefers metadata.json (written by make-void-kernel.sh).
2. Falls back to globbing: boot/vmlinux-* or vmlinuz-* (Alpine
fallback), boot/initramfs-*, lib/modules/<latest>.
The daemon's KernelImport copies kernel + optional initrd via
system.CopyFilePreferClone and modules via system.CopyDirContents
(no-sudo mode — catalog lives under ~/.local/state), computes SHA256
over the kernel, and writes the manifest via kernelcat.WriteLocal.
While wiring this up, fixed a latent bug in system.CopyDirContents:
filepath.Join(sourceDir, ".") silently drops the trailing dot, so
`cp -a source source/contents target/` was copying the whole source
directory (including its basename) instead of just its contents.
Replaced the join with a manual "/." suffix. imagemgr.StageBootArtifacts
(the only existing caller) silently benefits.
scripts/register-void-image.sh and scripts/register-alpine-image.sh
are rewritten to use `banger kernel import … && banger image register
--kernel-ref …` instead of the find-and-pass-paths dance. Preserves
the same user-facing commands and env vars.
Tests cover: metadata.json preference, glob fallback, Alpine vmlinuz
fallback, kernel-missing error, round-trip copy into the catalog, and
the --from required flag.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a read/write kernel catalog on disk without any network
dependency, so later phases (image register --kernel-ref, import, pull)
can build on a working foundation.
Layout: adds KernelsDir to paths.Layout, ensured under
~/.local/state/banger/kernels/. Each cataloged kernel lives at
<KernelsDir>/<name>/ with a manifest.json alongside vmlinux and optional
initrd.img / modules/.
New internal/kernelcat package owns the disk format:
- Entry (Name, Distro, Arch, KernelVersion, SHA256, Source, ImportedAt)
- ValidateName (alphanumeric + dots/hyphens/underscores, no traversal)
- ReadLocal / ListLocal / WriteLocal / DeleteLocal
- SumFile helper
The daemon exposes three RPC methods dispatched in daemon.go:
kernel.list, kernel.show, kernel.delete. Implementations live in a new
internal/daemon/kernels.go and are thin wrappers over kernelcat using
d.layout.KernelsDir.
CLI: new top-level `banger kernel` with list / show / rm subcommands
mirroring the image-command pattern (ensureDaemon, RPC call, table or
JSON output). No sudo required — kernel ops are user-space only.
Users can now manually populate ~/.local/state/banger/kernels/<name>/
and see it via `banger kernel list`. Phase 2 wires --kernel-ref into
image register; Phase 3 adds `banger kernel import`; Phase 4 adds
remote pulls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
