update: VMs survive banger update and rollback
Three load-bearing fixes that together let `banger update` (and its auto-rollback path) restart the helper + daemon without killing every running VM. New smoke scenarios prove the property end-to-end. Bug fixes: 1. Disable the firecracker SDK's signal-forwarding goroutine. The default ForwardSignals = [SIGINT, SIGQUIT, SIGTERM, SIGHUP, SIGABRT] installs a handler in the helper that propagates the helper's SIGTERM (sent by systemd on `systemctl stop bangerd- root.service`) to every running firecracker child. Set ForwardSignals to an empty (non-nil) slice so setupSignals short-circuits at len()==0. 2. Add SendSIGKILL=no to bangerd-root.service. KillMode=process limits the initial SIGTERM to the helper main, but systemd still SIGKILLs leftover cgroup processes during the FinalKillSignal stage unless SendSIGKILL=no. 3. Route restart-helper / restart-daemon / wait-daemon-ready failures through rollbackAndRestart instead of rollbackAndWrap. rollbackAndWrap restored .previous binaries but didn't re- restart the failed unit, leaving the helper dead with the rolled-back binary on disk after a failed update. Testing infrastructure (production binaries unaffected): - Hidden --manifest-url and --pubkey-file flags on `banger update` let the smoke harness redirect the updater at locally-built release artefacts. Marked Hidden in cobra; not advertised in --help. - FetchManifestFrom / VerifyBlobSignatureWithKey / FetchAndVerifySignatureWithKey export the existing logic against caller-supplied URL / pubkey. The default entry points still call them with the embedded canonical values. Smoke scenarios: - update_check: --check against fake manifest reports update available - update_to_unknown: --to v9.9.9 fails before any host mutation - update_no_root: refuses without sudo, install untouched - update_dry_run: stages + verifies, no swap, version unchanged - update_keeps_vm_alive: real swap to v0.smoke.0; same VM (same boot_id) answers SSH after the daemon restart - update_rollback_keeps_vm_alive: v0.smoke.broken-bangerd ships a bangerd that passes --check-migrations but exits 1 as the daemon. The post-swap `systemctl restart bangerd` fails, rollbackAndRestart fires, the .previous binaries are restored and re-restarted; the same VM still answers SSH afterwards - daemon_admin (separate prep): covers `banger daemon socket`, `bangerd --check-migrations --system`, `sudo banger daemon stop` The smoke release builder generates a fresh ECDSA P-256 keypair with openssl, signs SHA256SUMS cosign-compatibly, and serves artefacts from a backgrounded python http.server. verify_smoke_check_test.go pins the openssl/cosign signature equivalence so the smoke release builder can't silently drift. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
7e528f30b3
commit
2606bfbabb
8 changed files with 609 additions and 50 deletions
|
|
@ -75,15 +75,23 @@ type Release struct {
|
|||
// Release.
|
||||
const ManifestSchemaVersion = 1
|
||||
|
||||
// FetchManifest downloads the release manifest and validates its
|
||||
// shape. Returns an error if the server is unreachable, returns
|
||||
// non-2xx, exceeds the size cap, or the schema_version is newer
|
||||
// than this CLI knows.
|
||||
// FetchManifest downloads the release manifest from the embedded
|
||||
// canonical URL and validates its shape. Returns an error if the
|
||||
// server is unreachable, returns non-2xx, exceeds the size cap, or
|
||||
// the schema_version is newer than this CLI knows.
|
||||
func FetchManifest(ctx context.Context, client *http.Client) (Manifest, error) {
|
||||
return FetchManifestFrom(ctx, client, manifestURL)
|
||||
}
|
||||
|
||||
// FetchManifestFrom is FetchManifest against an explicit URL. Used by
|
||||
// the smoke suite (via `banger update --manifest-url …`) to drive the
|
||||
// updater against a locally-served fake manifest. Production callers
|
||||
// stick with FetchManifest.
|
||||
func FetchManifestFrom(ctx context.Context, client *http.Client, url string) (Manifest, error) {
|
||||
if client == nil {
|
||||
client = http.DefaultClient
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, manifestURL, nil)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return Manifest{}, err
|
||||
}
|
||||
|
|
|
|||
|
|
@ -61,18 +61,26 @@ var ErrSignatureRequired = errors.New("banger release public key is the placehol
|
|||
|
||||
// VerifyBlobSignature checks that sigBase64 is a valid cosign-blob
|
||||
// signature over body, made with the private counterpart of
|
||||
// BangerReleasePublicKey. cosign's blob signature format is a
|
||||
// base64-encoded ASN.1-DER ECDSA signature over SHA256(body) — that's
|
||||
// what the package's ecdsa.VerifyASN1 verifies natively.
|
||||
//
|
||||
// Refuses outright if the embedded public key is still the build-
|
||||
// time placeholder, so an unset key can't slip through as
|
||||
// "verification disabled."
|
||||
// BangerReleasePublicKey.
|
||||
func VerifyBlobSignature(body, sigBase64 []byte) error {
|
||||
if isPlaceholderKey(BangerReleasePublicKey) {
|
||||
return VerifyBlobSignatureWithKey(body, sigBase64, BangerReleasePublicKey)
|
||||
}
|
||||
|
||||
// VerifyBlobSignatureWithKey is VerifyBlobSignature against an
|
||||
// explicit PEM-encoded public key. Used by the smoke suite (via
|
||||
// `banger update --pubkey-file …`) so an end-to-end update test can
|
||||
// trust a locally-generated keypair without rebuilding the binary.
|
||||
//
|
||||
// Refuses outright if pubKeyPEM is the build-time placeholder so an
|
||||
// unset key can't slip through as "verification disabled".
|
||||
//
|
||||
// cosign's blob signature format is a base64-encoded ASN.1-DER ECDSA
|
||||
// signature over SHA256(body) — that's what ecdsa.VerifyASN1 takes.
|
||||
func VerifyBlobSignatureWithKey(body, sigBase64 []byte, pubKeyPEM string) error {
|
||||
if isPlaceholderKey(pubKeyPEM) {
|
||||
return ErrSignatureRequired
|
||||
}
|
||||
block, _ := pem.Decode([]byte(BangerReleasePublicKey))
|
||||
block, _ := pem.Decode([]byte(pubKeyPEM))
|
||||
if block == nil {
|
||||
return fmt.Errorf("decode banger release public key: no PEM block")
|
||||
}
|
||||
|
|
@ -96,15 +104,21 @@ func VerifyBlobSignature(body, sigBase64 []byte) error {
|
|||
}
|
||||
|
||||
// FetchAndVerifySignature pulls the SHA256SUMS.sig URL from the
|
||||
// release, downloads it (capped), and verifies it against
|
||||
// sumsBody. Returns nil on a clean pass, or an error describing
|
||||
// exactly why verification failed.
|
||||
// release, downloads it (capped), and verifies it against sumsBody.
|
||||
// Returns nil on a clean pass, or an error describing exactly why
|
||||
// verification failed.
|
||||
//
|
||||
// If release.SHA256SumsSigURL is empty, treat that as "release was
|
||||
// not signed" — refuse rather than silently proceeding. v0.1.0
|
||||
// requires every release to be cosign-signed; an unsigned release
|
||||
// is a manifest publishing bug we'd rather catch loudly.
|
||||
func FetchAndVerifySignature(ctx context.Context, client *http.Client, release Release, sumsBody []byte) error {
|
||||
return FetchAndVerifySignatureWithKey(ctx, client, release, sumsBody, BangerReleasePublicKey)
|
||||
}
|
||||
|
||||
// FetchAndVerifySignatureWithKey is FetchAndVerifySignature against
|
||||
// an explicit PEM-encoded public key.
|
||||
func FetchAndVerifySignatureWithKey(ctx context.Context, client *http.Client, release Release, sumsBody []byte, pubKeyPEM string) error {
|
||||
if strings.TrimSpace(release.SHA256SumsSigURL) == "" {
|
||||
return fmt.Errorf("release %s has no sha256sums_sig_url; refusing to install an unsigned release", release.Version)
|
||||
}
|
||||
|
|
@ -115,7 +129,7 @@ func FetchAndVerifySignature(ctx context.Context, client *http.Client, release R
|
|||
if err != nil {
|
||||
return fmt.Errorf("fetch signature: %w", err)
|
||||
}
|
||||
if err := VerifyBlobSignature(sumsBody, sig); err != nil {
|
||||
if err := VerifyBlobSignatureWithKey(sumsBody, sig, pubKeyPEM); err != nil {
|
||||
return fmt.Errorf("verify SHA256SUMS signature: %w", err)
|
||||
}
|
||||
return nil
|
||||
|
|
|
|||
54
internal/updater/verify_smoke_check_test.go
Normal file
54
internal/updater/verify_smoke_check_test.go
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
package updater
|
||||
|
||||
import (
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestVerifyBlobSignatureWithOpenSSL is a confidence test for the
|
||||
// smoke release-builder path: openssl's `dgst -sha256 -sign` produces
|
||||
// the exact same encoding cosign emits for blob signatures (base64
|
||||
// ASN.1 ECDSA over SHA256(body)). If this ever stops verifying, the
|
||||
// smoke update scenarios will silently skip the signature check —
|
||||
// catching it here avoids a heisenbug in scripts/smoke.sh.
|
||||
func TestVerifyBlobSignatureWithOpenSSL(t *testing.T) {
|
||||
if _, err := exec.LookPath("openssl"); err != nil {
|
||||
t.Skip("openssl not on PATH")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
keyPath := filepath.Join(dir, "cosign.key")
|
||||
pubPath := filepath.Join(dir, "cosign.pub")
|
||||
bodyPath := filepath.Join(dir, "body.txt")
|
||||
sigPath := filepath.Join(dir, "body.sig")
|
||||
|
||||
mustRun := func(name string, args ...string) {
|
||||
t.Helper()
|
||||
out, err := exec.Command(name, args...).CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("%s %v: %v\n%s", name, args, err, string(out))
|
||||
}
|
||||
}
|
||||
|
||||
mustRun("openssl", "ecparam", "-name", "prime256v1", "-genkey", "-noout", "-out", keyPath)
|
||||
mustRun("openssl", "ec", "-in", keyPath, "-pubout", "-out", pubPath)
|
||||
mustRun("sh", "-c", "printf 'banger smoke release sums\n' > "+bodyPath)
|
||||
mustRun("sh", "-c", "openssl dgst -sha256 -sign "+keyPath+" "+bodyPath+" | base64 -w0 > "+sigPath)
|
||||
|
||||
body := readFile(t, bodyPath)
|
||||
sig := readFile(t, sigPath)
|
||||
pub := readFile(t, pubPath)
|
||||
|
||||
if err := VerifyBlobSignatureWithKey(body, sig, string(pub)); err != nil {
|
||||
t.Fatalf("VerifyBlobSignatureWithKey: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func readFile(t *testing.T, p string) []byte {
|
||||
t.Helper()
|
||||
out, err := exec.Command("cat", p).Output()
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", p, err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue