thaloco/banger
1
Fork 0
Code Pull requests Releases Packages Activity Actions
banger/scripts/smoke.sh
Thales Maciel 672d7151e9
smoke: five more scenarios + fix exit-code propagation bug the new ones caught 
Five new smoke scenarios layered on top of the existing bare + workspace
vm-run tests:

  - exit-code propagation: `sh -c 'exit 42'` must rc=42
  - workspace dry-run: --dry-run lists tracked files without a VM
  - workspace --include-untracked: opt-in ships files outside the git
    index (regression guard on the security-default flip from review 4)
  - concurrent vm runs: two --rm invocations in parallel both succeed
    (stresses per-VM locks, createVMMu reservation window, tap pool)
  - invalid spec rejection: --vcpu 0 must fail with no VM row left
    behind (the "cleanup on partial failure" path the review flagged)

The exit-code scenario caught a real bug on first run:

  `banger vm run --rm -- sh -c 'exit 42'` returned rc=0, not 42.

Root cause in internal/cli/ssh.go's sshCommandArgs: extra args were
appended to the ssh argv verbatim, relying on ssh(1)'s implicit
space-join to deliver the remote command. That works for single
tokens (echo hello) but re-tokenises multi-word commands on the
remote side: `ssh host sh -c 'exit 42'` becomes remote
`sh -c exit 42`, where `42` is $0 for the already-completed `exit`,
and the exit code the user asked for is lost.

Fix: shell-quote every element of extra (`'sh'` `'-c'` `'exit 42'`)
and join them into a single trailing argv entry. ssh's space-join
then produces exactly the command the user typed on the remote
shell. TestSSHCommandArgs was updated to pin the quoting; the
existing TestRunVMRunCommandModePropagatesExitCode test needed a
one-word quote tweak (`false` → `'false'`).

Smoke run after fix passes all seven scenarios in ~2 min on warm
state. cmd/banger coverage jumped to 100% (the invalid-spec
scenario hits the error-reporting path that wasn't covered
before).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 19:37:07 -03:00

186 lines
8.5 KiB
Bash
Executable file
Raw Blame History

#!/usr/bin/env bash
#
# scripts/smoke.sh — end-to-end smoke suite for banger.
#
# Drives a real create → start → ssh → exec → delete cycle against
# real Firecracker + real KVM on the host. Intended as a pre-release
# gate: the Go unit + integration tests don't and can't cover the
# post-machine.Start path (socket ownership, guest boot, vsock agent
# wait, guest SSH, workspace prepare). If this suite fails, don't
# ship.
#
# State lives under $BANGER_SMOKE_XDG_DIR (set by `make smoke`,
# defaults to build/smoke/xdg). It's ISOLATED from the invoking
# user's real banger install via XDG_{CONFIG,STATE,CACHE,RUNTIME}
# overrides, but PERSISTED across runs — so the first smoke pulls
# the golden image, subsequent smokes reuse it. `make smoke-clean`
# wipes it.
#
# Invoked via `make smoke`, which sets the three env vars below.
# Don't run this directly unless you know they're set.
set -euo pipefail
log() { printf '[smoke] %s\n' "$*" >&2; }
die() { printf '[smoke] FAIL: %s\n' "$*" >&2; exit 1; }
: "${BANGER_SMOKE_BIN_DIR:?must point at the instrumented binary dir, set by make smoke}"
: "${BANGER_SMOKE_COVER_DIR:?must point at the coverage dir, set by make smoke}"
: "${BANGER_SMOKE_XDG_DIR:?must point at the isolated XDG root, set by make smoke}"
BANGER="$BANGER_SMOKE_BIN_DIR/banger"
BANGERD="$BANGER_SMOKE_BIN_DIR/bangerd"
VSOCK_AGENT="$BANGER_SMOKE_BIN_DIR/banger-vsock-agent"
for bin in "$BANGER" "$BANGERD" "$VSOCK_AGENT"; do
[[ -x "$bin" ]] || die "binary missing or not executable: $bin"
done
# Persistent XDG dirs (state, cache, config) so repeated smoke
# runs reuse the pulled golden image instead of re-downloading
# ~300MB each time. Runtime dir needs to be fresh per-run because
# it holds sockets the daemon cleans up on stop and refuses to
# reuse if any are stale.
mkdir -p \
"$BANGER_SMOKE_XDG_DIR/config" \
"$BANGER_SMOKE_XDG_DIR/state" \
"$BANGER_SMOKE_XDG_DIR/cache"
runtime_dir="$(mktemp -d -t banger-smoke-runtime-XXXXXX)"
# shellcheck disable=SC2064
trap "rm -rf '$runtime_dir'" EXIT
chmod 0700 "$runtime_dir"
export XDG_CONFIG_HOME="$BANGER_SMOKE_XDG_DIR/config"
export XDG_STATE_HOME="$BANGER_SMOKE_XDG_DIR/state"
export XDG_CACHE_HOME="$BANGER_SMOKE_XDG_DIR/cache"
export XDG_RUNTIME_DIR="$runtime_dir"
# Point banger at its companion binaries inside the smoke build.
export BANGER_DAEMON_BIN="$BANGERD"
export BANGER_VSOCK_AGENT_BIN="$VSOCK_AGENT"
# Instrumented binaries dump coverage here on clean exit.
export GOCOVERDIR="$BANGER_SMOKE_COVER_DIR"
mkdir -p "$GOCOVERDIR"
# Any smoke daemon left behind from a prior run that crashed mid-
# scenario would reuse the stale socket path and confuse
# ensureDaemon. Best-effort stop; ignore if nothing is running.
"$BANGER" daemon stop >/dev/null 2>&1 || true
# banger's vmDNS binds 127.0.0.1:42069 (UDP) hard. If the user's
# real (non-smoke) daemon is running, its listener holds the port
# and the smoke daemon's Open() fails before any scenario runs.
# Fail fast with an actionable message — don't guess whether to
# stop the user's daemon for them.
if command -v ss >/dev/null 2>&1 && ss -Huln 2>/dev/null | awk '{print $4}' | grep -q '[:.]42069$'; then
die 'port 127.0.0.1:42069 is already bound (likely your real banger daemon); stop it with `banger daemon stop` and re-run `make smoke`'
fi
# --- doctor -----------------------------------------------------------
log 'doctor: checking host readiness'
if ! "$BANGER" doctor; then
die 'doctor reported failures; fix the host before running smoke'
fi
# --- bare vm run ------------------------------------------------------
log "bare vm run: create + start + ssh + exec 'echo smoke-bare-ok' + --rm"
bare_out="$("$BANGER" vm run --rm -- echo smoke-bare-ok)" || die "bare vm run exit $?"
grep -q 'smoke-bare-ok' <<<"$bare_out" || die "bare vm run stdout missing marker: $bare_out"
# --- workspace vm run -------------------------------------------------
log 'workspace vm run: preparing a throwaway git repo'
repodir="$runtime_dir/fake-repo"
mkdir -p "$repodir"
(
cd "$repodir"
git init -q -b main
git config commit.gpgsign false
git config user.name smoke
git config user.email smoke@smoke
echo 'smoke-workspace-marker' > smoke-file.txt
git add .
git commit -q -m init
)
log "workspace vm run: create + start + workspace prepare + cat guest file + --rm"
ws_out="$("$BANGER" vm run --rm "$repodir" -- cat /root/repo/smoke-file.txt)" || die "workspace vm run exit $?"
grep -q 'smoke-workspace-marker' <<<"$ws_out" || die "workspace vm run didn't ship smoke-file.txt: $ws_out"
# --- command exit-code propagation ------------------------------------
# A non-zero exit from the guest command must surface as banger's own
# exit code. Regressions here are hard to catch any other way — the
# local Go tests don't cross the SSH boundary, and users expect their
# CI scripts that wrap `banger vm run` to fail when the thing inside
# the VM failed.
log 'exit-code propagation: guest `sh -c "exit 42"` must produce rc=42'
set +e
"$BANGER" vm run --rm -- sh -c 'exit 42'
rc=$?
set -e
[[ "$rc" -eq 42 ]] || die "exit-code propagation: got rc=$rc, want 42"
# --- workspace dry-run (no VM) ----------------------------------------
# Pure CLI-side path — no VM, no sudo, just the local git inspection
# against d.repoInspector. Fast; catches regressions in the preview
# output (file list shape, mode line) that the Go tests already pin
# but that could still be broken by a client-side wiring change.
log 'workspace dry-run: list tracked files without creating a VM'
dry_out="$("$BANGER" vm run --dry-run "$repodir")" || die "dry-run exit $?"
grep -q 'smoke-file.txt' <<<"$dry_out" || die "dry-run didn't list smoke-file.txt: $dry_out"
grep -q 'mode: tracked only' <<<"$dry_out" || die "dry-run mode line missing or wrong: $dry_out"
# --- workspace --include-untracked -----------------------------------
# The default is tracked-only (review cycle 4). Opt-in must ship
# untracked files too. Write one, run with --include-untracked, verify
# it reaches the guest.
log 'workspace --include-untracked: opt-in ships files outside the git index'
echo 'untracked-marker' > "$repodir/smoke-untracked.txt"
inc_out="$("$BANGER" vm run --rm --include-untracked "$repodir" -- cat /root/repo/smoke-untracked.txt)" || die "include-untracked vm run exit $?"
grep -q 'untracked-marker' <<<"$inc_out" || die "--include-untracked didn't ship the untracked file: $inc_out"
# Restore repo to tracked-only state for any later scenarios.
rm -f "$repodir/smoke-untracked.txt"
# --- concurrent vm runs -----------------------------------------------
# Stresses per-VM lock scoping, the tap pool warm-up path, and
# createVMMu's narrow reservation window. Two `vm run --rm` invocations
# that actually overlap should both succeed. A regression that
# serialises create path too aggressively would make this slow but
# still pass; a regression that breaks tap allocation or name
# uniqueness would fail one of them.
log 'concurrent vm runs: two --rm invocations must both succeed'
tmpA="$runtime_dir/concurrent-a.out"
tmpB="$runtime_dir/concurrent-b.out"
"$BANGER" vm run --rm -- echo smoke-concurrent-a > "$tmpA" 2>&1 &
pidA=$!
"$BANGER" vm run --rm -- echo smoke-concurrent-b > "$tmpB" 2>&1 &
pidB=$!
wait "$pidA" || die "concurrent VM A exited non-zero: $(cat "$tmpA")"
wait "$pidB" || die "concurrent VM B exited non-zero: $(cat "$tmpB")"
grep -q 'smoke-concurrent-a' "$tmpA" || die "concurrent VM A missing marker: $(cat "$tmpA")"
grep -q 'smoke-concurrent-b' "$tmpB" || die "concurrent VM B missing marker: $(cat "$tmpB")"
# --- invalid spec rejection + no artifact leak ------------------------
# Tests the negative-path create flow: a blatantly invalid VM spec
# must fail before any VM row is persisted. The review cycle flagged
# "cleanup on partial failure" as under-tested; this scenario pins
# that a rejected create doesn't leak a reservation we then have to
# clean up by hand.
log 'invalid spec rejection: --vcpu 0 must fail and leave no VM behind'
pre_vms="$("$BANGER" vm list --all 2>/dev/null | wc -l)"
set +e
"$BANGER" vm run --rm --vcpu 0 -- echo unused >/dev/null 2>&1
rc=$?
set -e
[[ "$rc" -ne 0 ]] || die 'invalid spec: vm run succeeded despite --vcpu 0'
post_vms="$("$BANGER" vm list --all 2>/dev/null | wc -l)"
[[ "$pre_vms" == "$post_vms" ]] || die "invalid spec leaked a VM row: pre=$pre_vms, post=$post_vms"
# --- daemon stop (flushes coverage) -----------------------------------
log 'stopping daemon so instrumented binaries flush coverage'
"$BANGER" daemon stop >/dev/null 2>&1 || true
# Give the daemon a moment to write its covdata pod before the trap
# tears down runtime_dir.
sleep 0.5
log 'all scenarios passed'
View git blame Copy permalink