Thales Maciel
115eec8576
`scripts/smoke.sh` was a 600-line linear script: no way to see what it covers without reading the whole thing, and no way to run a single scenario when iterating. Every iteration paid the full ~5-10 min suite, which made fast feedback loops painful enough to avoid the suite. Refactor into a registry + per-scenario functions: - Top-of-file SMOKE_SCENARIOS (ordered) + SMOKE_DESCS (one-line desc per scenario) + SMOKE_CLASS (pure / repodir / global) drive both listing and dispatch. The 21 existing scenario blocks become scenario_<name> functions. Bodies are the inline blocks verbatim, modulo the workspace fixture move described below. - New CLI: --list (cheap discovery, no install / no env-vars), --scenario NAME (or NAME,NAME,...), --jobs N (parallel dispatch), -h / --help. - New setup_fixtures runs once after the install/doctor/restart preamble and produces the throwaway git repo at $repodir that 'repodir'-class scenarios consume. Lifted out of scenario_workspace_run so single- scenario invocations (e.g. --scenario workspace_dryrun) get the fixture even when the scenario that historically built it isn't selected. - Wipe ~/.local/state/banger/ssh/known_hosts in the install preamble. `system uninstall --purge` clears /var/lib/banger but the user-side known_hosts persists by design — and smoke creates VMs that reuse guest IPs (172.16.0.2 etc.) with fresh host keys every run, so a leftover entry trips StrictHostKeyChecking and the daemon's wait- for-ssh sees only timeouts. This was the real cause of the "guest ssh did not come up" flakes that surface across smoke iterations. Parallel dispatch: - --jobs N opts into a slot-limited pool: 'pure' scenarios fan out as individual jobs; 'repodir' scenarios fuse into a single serial chain (since they mutate $repodir in registry order); 'global' scenarios run serially after the pool, one at a time. - Cap is min(N, 8) — each parallel slot runs an 8 GiB VM, so RAM is the binding constraint. - Parallel-mode stdout/stderr per scenario buffer to per-scenario logs and emit one PASS/FAIL line on completion; on FAIL the buffer is dumped. Serial mode (--jobs 1, the default) keeps stdout unbuffered exactly as before. - Parallelism is documented as experimental in --help: it surfaces real daemon-side concurrency bugs (image auto-pull manifest race, work-seed-refresh race on the shared work-seed.ext4) that don't appear in serial mode and that need their own fix in the daemon. Serial (--jobs 1) is the reliable path; --jobs N is for fast- iteration dev work where occasional re-runs are acceptable. Exit codes: 0 ok, 1 assertion failed, 2 usage error (unknown scenario, missing SCENARIO=), 77 explicit selection skipped (NAT when sudo iptables is unavailable AND nat is the only selected scenario; soft-skip otherwise). Makefile additions: - `make smoke-list` — cheap discovery, no smoke-build dep, no env vars. - `make smoke-one SCENARIO=name` — single-scenario run, full preamble. MAKECMDGOALS guard catches missing SCENARIO= before any rebuild. - `make smoke JOBS=N` — passes through to the script's --jobs N. - Help text covers all three. Verified: serial full suite passes 21/21 in ~140s on this host; make smoke-one SCENARIO=workspace_restart runs the recently-added regression test alone in ~50s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
254 lines
12 KiB
Makefile
254 lines
12 KiB
Makefile
SHELL := /usr/bin/env bash
|
|
|
|
GO ?= go
|
|
GOFMT ?= gofmt
|
|
INSTALL ?= install
|
|
PREFIX ?= $(HOME)/.local
|
|
BINDIR ?= $(PREFIX)/bin
|
|
LIBDIR ?= $(PREFIX)/lib
|
|
DESTDIR ?=
|
|
BUILD_DIR ?= build
|
|
BUILD_BIN_DIR ?= $(BUILD_DIR)/bin
|
|
BUILD_MANUAL_DIR ?= $(BUILD_DIR)/manual
|
|
BANGER_BIN ?= $(BUILD_BIN_DIR)/banger
|
|
BANGERD_BIN ?= $(BUILD_BIN_DIR)/bangerd
|
|
VSOCK_AGENT_BIN ?= $(BUILD_BIN_DIR)/banger-vsock-agent
|
|
BINARIES := $(BANGER_BIN) $(BANGERD_BIN) $(VSOCK_AGENT_BIN)
|
|
GO_SOURCES := $(shell find cmd internal -type f -name '*.go' | sort)
|
|
# BUILD_INPUTS is everything that can change a binary's bytes: Go sources
|
|
# plus embedded assets (catalog.json, future static files). Listing
|
|
# everything is cheaper than missing a rebuild — go's own cache absorbs
|
|
# any redundant invocations.
|
|
BUILD_INPUTS := $(shell find cmd internal -type f | sort)
|
|
SHELL_SOURCES := $(shell find scripts -type f -name '*.sh' | sort)
|
|
SMOKE_DIR := $(BUILD_DIR)/smoke
|
|
SMOKE_BIN_DIR := $(SMOKE_DIR)/bin
|
|
SMOKE_COVER_DIR := $(SMOKE_DIR)/covdata
|
|
SMOKE_XDG_DIR := $(SMOKE_DIR)/xdg
|
|
SMOKE_SCRIPT := scripts/smoke.sh
|
|
VERSION ?= $(shell git describe --tags --exact-match 2>/dev/null || echo dev)
|
|
COMMIT ?= $(shell git rev-parse --verify HEAD 2>/dev/null || echo unknown)
|
|
BUILT_AT ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
GO_LDFLAGS := -X banger/internal/buildinfo.Version=$(VERSION) -X banger/internal/buildinfo.Commit=$(COMMIT) -X banger/internal/buildinfo.BuiltAt=$(BUILT_AT)
|
|
|
|
.DEFAULT_GOAL := help
|
|
|
|
# `make smoke-one` requires SCENARIO=. Validate before any prerequisite
|
|
# (notably smoke-build) so a typo'd invocation doesn't pay for a Go
|
|
# rebuild before learning it's wrong.
|
|
ifneq (,$(filter smoke-one,$(MAKECMDGOALS)))
|
|
ifndef SCENARIO
|
|
$(error smoke-one needs SCENARIO=name (see `make smoke-list` for names))
|
|
endif
|
|
endif
|
|
|
|
.PHONY: help build banger bangerd test fmt tidy clean install uninstall lint lint-go lint-shell coverage coverage-html coverage-total coverage-combined coverage-combined-html smoke smoke-build smoke-list smoke-one smoke-coverage-html smoke-clean smoke-fresh
|
|
|
|
help:
|
|
@printf '%s\n' \
|
|
'Targets:' \
|
|
' make build Build ./build/bin/banger, ./build/bin/bangerd, and ./build/bin/banger-vsock-agent' \
|
|
' make install Build and install banger, bangerd, and the companion vsock helper' \
|
|
' make uninstall Stop the daemon and remove installed binaries (leaves user state by default)' \
|
|
' make test Run go test ./...' \
|
|
' make coverage Run tests with coverage; print per-package + total' \
|
|
' make coverage-html Open a browsable per-line HTML report (writes coverage.html)' \
|
|
' make coverage-total Print just the total statement coverage (for scripts/CI)' \
|
|
' make coverage-combined Merge unit-test + smoke covdata; print per-package + total' \
|
|
' make coverage-combined-html HTML report of the merged unit+smoke coverage' \
|
|
' make lint Run gofmt + go vet + shellcheck (errors)' \
|
|
' make fmt Format Go sources under cmd/ and internal/' \
|
|
' make tidy Run go mod tidy' \
|
|
' make clean Remove built Go binaries and coverage artefacts' \
|
|
' make smoke Build instrumented binaries, run the supported systemd smoke suite, report coverage (needs KVM + sudo)' \
|
|
' make smoke JOBS=N Same, but dispatch parallel-safe scenarios across N slots (1-8; default 1)' \
|
|
' make smoke-list Print the list of smoke scenarios with descriptions (no build, no install)' \
|
|
' make smoke-one SCENARIO=NAME Run a single smoke scenario (still does the install preamble)' \
|
|
' make smoke-fresh smoke-clean + smoke — purges stale smoke-owned installs before a clean supported-path run' \
|
|
' make smoke-coverage-html HTML coverage report from the last smoke run' \
|
|
' make smoke-clean Remove the smoke build tree and purge any stale smoke-owned system install'
|
|
|
|
build: $(BINARIES)
|
|
|
|
$(BANGER_BIN): $(BUILD_INPUTS) go.mod go.sum
|
|
mkdir -p "$(BUILD_BIN_DIR)"
|
|
$(GO) build -ldflags '$(GO_LDFLAGS)' -o "$(BANGER_BIN)" ./cmd/banger
|
|
|
|
$(BANGERD_BIN): $(BUILD_INPUTS) go.mod go.sum
|
|
mkdir -p "$(BUILD_BIN_DIR)"
|
|
$(GO) build -ldflags '$(GO_LDFLAGS)' -o "$(BANGERD_BIN)" ./cmd/bangerd
|
|
|
|
$(VSOCK_AGENT_BIN): $(BUILD_INPUTS) go.mod go.sum
|
|
mkdir -p "$(BUILD_BIN_DIR)"
|
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GO) build -ldflags '$(GO_LDFLAGS)' -o "$(VSOCK_AGENT_BIN)" ./cmd/banger-vsock-agent
|
|
|
|
test:
|
|
$(GO) test ./...
|
|
|
|
# Coverage targets use -coverpkg=./... so packages without their own
|
|
# tests still get counted when another package exercises them (common
|
|
# for daemon/* subpackages). coverage.out is gitignored.
|
|
coverage:
|
|
$(GO) test -coverpkg=./... -coverprofile=coverage.out ./...
|
|
@echo ''
|
|
@echo 'Per-package:'
|
|
@$(GO) tool cover -func=coverage.out | awk -F'\t+' '/^total:/ {total=$$NF; next} {pkg=$$1; sub("banger/", "", pkg); sub("/[^/]+:[0-9]+:$$", "", pkg); pkgs[pkg]+=1; covered[pkg]+=$$NF+0} END {for (p in pkgs) printf " %-40s %.1f%% (avg of %d funcs)\n", p, covered[p]/pkgs[p], pkgs[p] | "sort"; print ""; print "Total statement coverage:", total}'
|
|
|
|
coverage-html: coverage
|
|
$(GO) tool cover -html=coverage.out -o coverage.html
|
|
@echo 'wrote coverage.html'
|
|
|
|
coverage-total:
|
|
@$(GO) test -coverpkg=./... -coverprofile=coverage.out ./... >/dev/null 2>&1 && $(GO) tool cover -func=coverage.out | awk '/^total:/ {print $$NF}'
|
|
|
|
# coverage-combined unions unit-test coverage and smoke coverage into
|
|
# one report. Unit tests cover pure-Go logic (error branches, parsing,
|
|
# handler wiring); smoke covers the real sudo / firecracker / dm-snap
|
|
# paths that unit tests physically can't reach. Separately each tells
|
|
# half the story. Merged, this is the single "what's not being
|
|
# exercised at all" view.
|
|
#
|
|
# Requires an up-to-date smoke run (the target depends on smoke-build
|
|
# to rebuild instrumented binaries; re-run `make smoke` yourself if
|
|
# scenarios changed). Modes must match; smoke uses the default 'set',
|
|
# so the unit run below drops the default 'atomic' for alignment.
|
|
COMBINED_COVER_DIR := $(BUILD_DIR)/combined
|
|
UNIT_COVER_DIR := $(BUILD_DIR)/unit/covdata
|
|
coverage-combined:
|
|
@test -d "$(SMOKE_COVER_DIR)" && test "$$(ls -A $(SMOKE_COVER_DIR) 2>/dev/null)" || { \
|
|
echo 'no smoke covdata at $(SMOKE_COVER_DIR); run `make smoke` first' >&2; exit 1; \
|
|
}
|
|
rm -rf "$(UNIT_COVER_DIR)" "$(COMBINED_COVER_DIR)"
|
|
mkdir -p "$(UNIT_COVER_DIR)" "$(COMBINED_COVER_DIR)"
|
|
$(GO) test -cover -coverpkg=./... ./... -args -test.gocoverdir="$(abspath $(UNIT_COVER_DIR))" >/dev/null
|
|
$(GO) tool covdata merge -i="$(UNIT_COVER_DIR),$(SMOKE_COVER_DIR)" -o="$(COMBINED_COVER_DIR)"
|
|
$(GO) tool covdata textfmt -i="$(COMBINED_COVER_DIR)" -o="$(BUILD_DIR)/combined.cover.out"
|
|
@echo ''
|
|
@echo 'Per-package (merged unit + smoke):'
|
|
@$(GO) tool cover -func="$(BUILD_DIR)/combined.cover.out" | awk -F'\t+' '/^total:/ {total=$$NF; next} {pkg=$$1; sub("banger/", "", pkg); sub("/[^/]+:[0-9]+:$$", "", pkg); pkgs[pkg]+=1; covered[pkg]+=$$NF+0} END {for (p in pkgs) printf " %-40s %.1f%% (avg of %d funcs)\n", p, covered[p]/pkgs[p], pkgs[p] | "sort"; print ""; print "Total statement coverage:", total}'
|
|
|
|
coverage-combined-html: coverage-combined
|
|
$(GO) tool cover -html="$(BUILD_DIR)/combined.cover.out" -o "$(BUILD_DIR)/combined.cover.html"
|
|
@echo 'wrote $(BUILD_DIR)/combined.cover.html'
|
|
|
|
lint: lint-go lint-shell
|
|
|
|
lint-go:
|
|
@unformatted="$$($(GOFMT) -l $(GO_SOURCES))"; \
|
|
if [ -n "$$unformatted" ]; then \
|
|
printf 'gofmt: the following files are not formatted:\n%s\n' "$$unformatted" >&2; \
|
|
exit 1; \
|
|
fi
|
|
$(GO) vet ./...
|
|
|
|
lint-shell:
|
|
@command -v shellcheck >/dev/null 2>&1 || { echo 'shellcheck is required for make lint-shell' >&2; exit 1; }
|
|
shellcheck --severity=error $(SHELL_SOURCES)
|
|
|
|
fmt:
|
|
$(GOFMT) -w $(GO_SOURCES)
|
|
|
|
tidy:
|
|
$(GO) mod tidy
|
|
|
|
clean:
|
|
rm -rf "$(BUILD_BIN_DIR)" coverage.out coverage.html
|
|
|
|
# Smoke test suite. Builds the three banger binaries with -cover
|
|
# instrumentation under $(SMOKE_BIN_DIR), installs them as temporary
|
|
# bangerd.service + bangerd-root.service, runs scripts/smoke.sh, copies
|
|
# service covdata out of /var/lib/banger, then purges the smoke-owned
|
|
# install on exit.
|
|
#
|
|
# Unlike the old per-user daemon path, this touches global systemd
|
|
# state. The smoke script refuses to overwrite a pre-existing non-smoke
|
|
# install and uses a marker file so `make smoke-clean` can recover a
|
|
# stale smoke-owned install after an interrupted run.
|
|
#
|
|
# Requires a KVM-capable Linux host with sudo. This is a pre-release
|
|
# gate, not CI — the Go test suite is what runs everywhere.
|
|
smoke-build: $(SMOKE_BIN_DIR)/.built
|
|
|
|
$(SMOKE_BIN_DIR)/.built: $(BUILD_INPUTS) go.mod go.sum
|
|
mkdir -p "$(SMOKE_BIN_DIR)"
|
|
$(GO) build -cover -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/banger" ./cmd/banger
|
|
$(GO) build -cover -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/bangerd" ./cmd/bangerd
|
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GO) build -ldflags '$(GO_LDFLAGS)' -o "$(SMOKE_BIN_DIR)/banger-vsock-agent" ./cmd/banger-vsock-agent
|
|
touch "$@"
|
|
|
|
smoke: smoke-build
|
|
rm -rf "$(SMOKE_COVER_DIR)"
|
|
mkdir -p "$(SMOKE_COVER_DIR)" "$(SMOKE_XDG_DIR)"
|
|
BANGER_SMOKE_BIN_DIR="$(abspath $(SMOKE_BIN_DIR))" \
|
|
BANGER_SMOKE_COVER_DIR="$(abspath $(SMOKE_COVER_DIR))" \
|
|
BANGER_SMOKE_XDG_DIR="$(abspath $(SMOKE_XDG_DIR))" \
|
|
bash "$(SMOKE_SCRIPT)" $(if $(JOBS),--jobs $(JOBS))
|
|
@echo ''
|
|
@echo 'Smoke coverage:'
|
|
@$(GO) tool covdata percent -i="$(SMOKE_COVER_DIR)"
|
|
|
|
# smoke-list is intentionally cheap: no smoke-build dep, no env vars.
|
|
# The script's --list path short-circuits before any side-effect or
|
|
# env validation, so this works on a fresh checkout.
|
|
smoke-list:
|
|
@bash "$(SMOKE_SCRIPT)" --list
|
|
|
|
# smoke-one runs one scenario (or a comma-separated list) with the same
|
|
# install preamble as the full suite. Useful when iterating on a specific
|
|
# scenario — see `make smoke-list` for names.
|
|
smoke-one: smoke-build
|
|
rm -rf "$(SMOKE_COVER_DIR)"
|
|
mkdir -p "$(SMOKE_COVER_DIR)" "$(SMOKE_XDG_DIR)"
|
|
BANGER_SMOKE_BIN_DIR="$(abspath $(SMOKE_BIN_DIR))" \
|
|
BANGER_SMOKE_COVER_DIR="$(abspath $(SMOKE_COVER_DIR))" \
|
|
BANGER_SMOKE_XDG_DIR="$(abspath $(SMOKE_XDG_DIR))" \
|
|
bash "$(SMOKE_SCRIPT)" --scenario "$(SCENARIO)"
|
|
|
|
smoke-coverage-html: smoke
|
|
$(GO) tool covdata textfmt -i="$(SMOKE_COVER_DIR)" -o="$(SMOKE_DIR)/cover.out"
|
|
$(GO) tool cover -html="$(SMOKE_DIR)/cover.out" -o "$(SMOKE_DIR)/cover.html"
|
|
@echo 'wrote $(SMOKE_DIR)/cover.html'
|
|
|
|
smoke-clean:
|
|
@if sudo test -f /etc/banger/.smoke-owned; then \
|
|
bin=''; \
|
|
if [ -x "$(SMOKE_BIN_DIR)/banger" ]; then \
|
|
bin="$(abspath $(SMOKE_BIN_DIR))/banger"; \
|
|
elif [ -x "$(BANGER_BIN)" ]; then \
|
|
bin="$(abspath $(BANGER_BIN))"; \
|
|
elif [ -x /usr/local/bin/banger ]; then \
|
|
bin=/usr/local/bin/banger; \
|
|
fi; \
|
|
if [ -n "$$bin" ]; then \
|
|
sudo "$$bin" system uninstall --purge >/dev/null 2>&1 || true; \
|
|
fi; \
|
|
fi
|
|
rm -rf "$(SMOKE_DIR)"
|
|
|
|
# smoke-fresh wipes the instrumented build tree, purges any stale
|
|
# smoke-owned install, and then runs the supported-path smoke suite
|
|
# from scratch.
|
|
smoke-fresh: smoke-clean smoke
|
|
|
|
install: build
|
|
mkdir -p "$(DESTDIR)$(BINDIR)"
|
|
mkdir -p "$(DESTDIR)$(LIBDIR)/banger"
|
|
$(INSTALL) -m 0755 "$(BANGER_BIN)" "$(DESTDIR)$(BINDIR)/banger"
|
|
$(INSTALL) -m 0755 "$(BANGERD_BIN)" "$(DESTDIR)$(BINDIR)/bangerd"
|
|
$(INSTALL) -m 0755 "$(VSOCK_AGENT_BIN)" "$(DESTDIR)$(LIBDIR)/banger/banger-vsock-agent"
|
|
|
|
# uninstall stops a running daemon (if any) and removes the installed
|
|
# binaries. It does NOT touch user data (config, SSH keys, VM state,
|
|
# image/kernel caches) — rm -rf those paths manually if wanted; they
|
|
# are printed for convenience.
|
|
uninstall:
|
|
@if [ -x "$(DESTDIR)$(BINDIR)/banger" ]; then \
|
|
"$(DESTDIR)$(BINDIR)/banger" daemon stop >/dev/null 2>&1 || true; \
|
|
fi
|
|
rm -f "$(DESTDIR)$(BINDIR)/banger" "$(DESTDIR)$(BINDIR)/bangerd"
|
|
rm -rf "$(DESTDIR)$(LIBDIR)/banger"
|
|
@printf '\nRemoved binaries. User data is preserved at:\n'
|
|
@printf ' ~/.config/banger/ (config, ssh keys)\n'
|
|
@printf ' ~/.local/state/banger/ (VMs, images, kernels, db, logs)\n'
|
|
@printf ' ~/.cache/banger/ (OCI layer cache)\n'
|
|
@printf '\nDelete those paths manually if you want a full purge.\n'
|