From 49c5c862b24352ba3ddeb90711bf514d3d293844 Mon Sep 17 00:00:00 2001 From: Thales Maciel Date: Sat, 18 Apr 2026 14:59:01 -0300 Subject: [PATCH] golden image: fix systemd boot + sshd startup Three fixes discovered during end-to-end boot testing on Firecracker: - Install udev + dbus alongside systemd. Both are Recommends of the systemd package, skipped by --no-install-recommends. Without udev, systemd never activates device units (dev-vdb.device stays inactive even after the kernel enumerates /dev/vdb) and the work-disk mount hangs forever. dbus is required by a growing set of services (logind, systemd-resolved shim, etc.). - Ship /usr/lib/tmpfiles.d/sshd.conf creating /run/sshd. Debian's openssh-server package doesn't ship one, and ssh.service's own RuntimeDirectory=sshd fires too late for the ExecStartPre config check, which blows up with 'Missing privilege separation directory'. The tmpfiles entry runs in systemd-tmpfiles-setup.service well before ssh.service starts. - Rewrite the ssh.service drop-in to reset the main unit's ExecStartPre list. Debian ships `sshd -t` as ExecStartPre #1; that fails without host keys and terminates the service before our `ssh-keygen -A` fires. Reset + re-add in the correct order: mkdir, keygen, then the test. StandardOutput/Error=journal+console on ssh.service so future sshd failures surface in the firecracker console log too, not only in the (unreachable) guest journal. Co-Authored-By: Claude Opus 4.7 (1M context) --- images/golden/Dockerfile | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/images/golden/Dockerfile b/images/golden/Dockerfile index 5723ede..c75296c 100644 --- a/images/golden/Dockerfile +++ b/images/golden/Dockerfile @@ -15,12 +15,21 @@ ENV DEBIAN_FRONTEND=noninteractive \ LC_ALL=C.UTF-8 # -------- 1. ESSENTIAL -------- -# Banger needs: an init (systemd), sshd (the only control channel), -# TLS roots + curl (first-boot installs + mise installer), iproute2 -# (debugging; `ip` is still useful even when the kernel sets IP via cmdline). +# Banger needs: an init (systemd + udev + dbus), sshd (the only +# control channel), TLS roots + curl (first-boot installs + mise +# installer), iproute2 (debugging; `ip` is still useful even when +# the kernel sets IP via cmdline). +# +# udev is a Recommends of the systemd package on Debian. With +# --no-install-recommends it's skipped — and without it systemd never +# activates device units, so fstab mounts of /dev/vdb (banger's work +# disk) hang forever waiting for a device that is already enumerated +# by the kernel but never "seen" by systemd. dbus gets the same +# treatment for the same reason (system-bus-ness services wedge +# without it). RUN apt-get update \ && apt-get install -y --no-install-recommends \ - systemd systemd-sysv \ + systemd systemd-sysv udev dbus \ openssh-server \ ca-certificates \ curl \ @@ -78,11 +87,29 @@ RUN ln -s /usr/bin/fdfind /usr/local/bin/fd # - SSH host keys: removed here; a ssh.service drop-in (below) runs # `ssh-keygen -A` before sshd so the VM's first boot generates a # unique set. +# - /run/sshd tmpfiles entry: Debian's openssh-server package doesn't +# ship one, and ssh.service's own `RuntimeDirectory=sshd` fires too +# late for the ExecStartPre config test, so sshd -t blows up with +# "Missing privilege separation directory: /run/sshd" before the +# daemon ever starts. Creating the dir via tmpfiles.d runs early in +# systemd-tmpfiles-setup, well before ssh.service kicks off. RUN : > /etc/machine-id \ && rm -f /etc/ssh/ssh_host_*_key /etc/ssh/ssh_host_*_key.pub \ && install -d /etc/systemd/system/ssh.service.d \ - && printf '[Service]\nExecStartPre=-/usr/bin/ssh-keygen -A\n' \ - > /etc/systemd/system/ssh.service.d/regen-host-keys.conf + && printf '%s\n' \ + '[Service]' \ + '# Reset main unit ExecStartPre list: Debian ships `sshd -t` as' \ + '# the first ExecStartPre, which fails on missing host keys and' \ + '# short-circuits the service before ours gets a chance to run.' \ + 'ExecStartPre=' \ + 'ExecStartPre=/usr/bin/mkdir -p /run/sshd' \ + 'ExecStartPre=/usr/bin/ssh-keygen -A' \ + 'ExecStartPre=/usr/sbin/sshd -t' \ + 'StandardOutput=journal+console' \ + 'StandardError=journal+console' \ + > /etc/systemd/system/ssh.service.d/banger.conf \ + && rm -f /etc/systemd/system/ssh.service.d/regen-host-keys.conf \ + && printf 'd /run/sshd 0755 root root -\n' > /usr/lib/tmpfiles.d/sshd.conf # No CMD / ENTRYPOINT: banger boots this via systemd as PID 1 after # first-boot, not via `docker run`.