make smoke: end-to-end boot suite with coverage from real VM runs
The unit + integration tests can't cross machine.Start — the SDK
boundary would need a fake firecracker that reimplements the
control-plane HTTP API, and the ongoing maintenance cost of keeping
that fake honest with upstream kills the value. Instead, add a
pre-release smoke target that drives REAL Firecracker + real KVM,
captures coverage from the -cover-instrumented binaries, and
surfaces per-package deltas so regressions in the boot path don't
ship silently.
scripts/smoke.sh:
- Isolated XDG_{CONFIG,STATE,CACHE,RUNTIME} so the smoke run can't
touch real user state (state/cache persist under build/smoke/xdg
for fast reruns; runtime is mktemp'd fresh per-run because
sockets can't be reused)
- Preflight: `banger doctor` must pass; UDP :42069 must be free
(otherwise the user's real daemon is up and the smoke daemon
can't bind its DNS listener — fail with an actionable message)
- Scenario 1 — bare: `banger vm run --rm -- echo smoke-bare-ok`
exercises create → start → socket ownership chown → machine.Start
→ SDK waitForSocket race → vsock agent readiness → guest SSH
wait → exec → cleanup → delete
- Scenario 2 — workspace: creates a throwaway git repo, runs
`banger vm run --rm <repo> -- cat /root/repo/smoke-file.txt`,
verifies the tracked file reached the guest (exercises
workDisk capability PrepareHost + workspace.prepare)
- `banger daemon stop` at the end so instrumented binaries flush
GOCOVERDIR pods before the script exits
Makefile additions:
- smoke-build: builds banger/bangerd under build/smoke/bin/ with
`go build -cover`
- smoke: runs the script with GOCOVERDIR set, reports per-package
coverage via `go tool covdata percent`
- smoke-coverage-html: textfmt + go tool cover for a browsable
report
- smoke-clean: nukes build/smoke/ including the persisted XDG
state
Bonus fix uncovered during the first smoke run: doctor treated a
missing state.db as a FAIL ("out of memory" from SQLite
SQLITE_CANTOPEN), which red-flagged every fresh install. Split
the store check: DB file absent → PASS with "will be created on
first daemon start" detail; DB present but unreadable → FAIL as
before. New TestDoctorReport_StoreMissingSurfacesAsPassForFreshInstall
pins the behaviour.
Concrete coverage delta from the first successful smoke run
(compared to `make coverage-total`'s unit-test-only 37.8%):
internal/firecracker 43.6% → 75.0%
internal/daemon/workspace 33.8% → 60.8%
internal/store 40.1% → 56.3%
internal/guest 63.7% → 57.4% (different mix: smoke
exercises real SSH;
unit tests cover more
error branches)
The packages the review flagged are the ones that moved most —
which is the point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
52612b516b
commit
5f81332b0a
5 changed files with 221 additions and 18 deletions
50
Makefile
50
Makefile
|
|
@ -21,6 +21,11 @@ GO_SOURCES := $(shell find cmd internal -type f -name '*.go' | sort)
|
|||
# any redundant invocations.
|
||||
BUILD_INPUTS := $(shell find cmd internal -type f | sort)
|
||||
SHELL_SOURCES := $(shell find scripts -type f -name '*.sh' | sort)
|
||||
SMOKE_DIR := $(BUILD_DIR)/smoke
|
||||
SMOKE_BIN_DIR := $(SMOKE_DIR)/bin
|
||||
SMOKE_COVER_DIR := $(SMOKE_DIR)/covdata
|
||||
SMOKE_XDG_DIR := $(SMOKE_DIR)/xdg
|
||||
SMOKE_SCRIPT := scripts/smoke.sh
|
||||
VERSION ?= $(shell git describe --tags --exact-match 2>/dev/null || echo dev)
|
||||
COMMIT ?= $(shell git rev-parse --verify HEAD 2>/dev/null || echo unknown)
|
||||
BUILT_AT ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||
|
|
@ -28,7 +33,7 @@ GO_LDFLAGS := -X banger/internal/buildinfo.Version=$(VERSION) -X banger/internal
|
|||
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
.PHONY: help build banger bangerd test fmt tidy clean install uninstall lint lint-go lint-shell coverage coverage-html coverage-total
|
||||
.PHONY: help build banger bangerd test fmt tidy clean install uninstall lint lint-go lint-shell coverage coverage-html coverage-total smoke smoke-build smoke-coverage-html smoke-clean
|
||||
|
||||
help:
|
||||
@printf '%s\n' \
|
||||
|
|
@ -43,7 +48,10 @@ help:
|
|||
' make lint Run gofmt + go vet + shellcheck (errors)' \
|
||||
' make fmt Format Go sources under cmd/ and internal/' \
|
||||
' make tidy Run go mod tidy' \
|
||||
' make clean Remove built Go binaries and coverage artefacts'
|
||||
' make clean Remove built Go binaries and coverage artefacts' \
|
||||
' make smoke Build instrumented binaries, run scripts/smoke.sh, report coverage (needs KVM + sudo)' \
|
||||
' make smoke-coverage-html HTML coverage report from the last smoke run' \
|
||||
' make smoke-clean Remove the smoke build tree'
|
||||
|
||||
build: $(BINARIES)
|
||||
|
||||
|
|
@ -101,6 +109,44 @@ tidy:
|
|||
clean:
|
||||
rm -rf "$(BUILD_BIN_DIR)" coverage.out coverage.html
|
||||
|
||||
# Smoke test suite. Builds the three banger binaries with -cover
|
||||
# instrumentation under $(SMOKE_BIN_DIR), runs scripts/smoke.sh
|
||||
# with GOCOVERDIR pointed at $(SMOKE_COVER_DIR), and prints the
|
||||
# resulting coverage. The smoke script fully isolates state via
|
||||
# XDG_* env vars pointing at a mktemp'd root, so the invoking
|
||||
# user's real banger install stays untouched.
|
||||
#
|
||||
# Requires a KVM-capable Linux host with sudo; fails fast via
|
||||
# `banger doctor` when either is missing. This is a pre-release
|
||||
# gate, not CI — the Go test suite is what runs everywhere.
|
||||
smoke-build: $(SMOKE_BIN_DIR)/.built
|
||||
|
||||
$(SMOKE_BIN_DIR)/.built: $(BUILD_INPUTS) go.mod go.sum
|
||||
mkdir -p "$(SMOKE_BIN_DIR)"
|
||||
$(GO) build -cover -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/banger" ./cmd/banger
|
||||
$(GO) build -cover -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/bangerd" ./cmd/bangerd
|
||||
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GO) build -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/banger-vsock-agent" ./cmd/banger-vsock-agent
|
||||
touch "$@"
|
||||
|
||||
smoke: smoke-build
|
||||
rm -rf "$(SMOKE_COVER_DIR)"
|
||||
mkdir -p "$(SMOKE_COVER_DIR)" "$(SMOKE_XDG_DIR)"
|
||||
BANGER_SMOKE_BIN_DIR="$(abspath $(SMOKE_BIN_DIR))" \
|
||||
BANGER_SMOKE_COVER_DIR="$(abspath $(SMOKE_COVER_DIR))" \
|
||||
BANGER_SMOKE_XDG_DIR="$(abspath $(SMOKE_XDG_DIR))" \
|
||||
bash "$(SMOKE_SCRIPT)"
|
||||
@echo ''
|
||||
@echo 'Smoke coverage:'
|
||||
@$(GO) tool covdata percent -i="$(SMOKE_COVER_DIR)"
|
||||
|
||||
smoke-coverage-html: smoke
|
||||
$(GO) tool covdata textfmt -i="$(SMOKE_COVER_DIR)" -o="$(SMOKE_DIR)/cover.out"
|
||||
$(GO) tool cover -html="$(SMOKE_DIR)/cover.out" -o "$(SMOKE_DIR)/cover.html"
|
||||
@echo 'wrote $(SMOKE_DIR)/cover.html'
|
||||
|
||||
smoke-clean:
|
||||
rm -rf "$(SMOKE_DIR)"
|
||||
|
||||
install: build
|
||||
mkdir -p "$(DESTDIR)$(BINDIR)"
|
||||
mkdir -p "$(DESTDIR)$(LIBDIR)/banger"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue