Speed up VM create with work seeds
Beat VM create wall time without changing VM semantics. Generate a work-seed ext4 sidecar during image builds and rootfs rebuilds, then clone and resize that seed for each new VM instead of rebuilding /root from scratch. Plumb the new seed artifact through config, runtime metadata, store state, runtime-bundle defaults, doctor checks, and default-image reconciliation so older images still fall back cleanly. Add a daemon TAP pool to keep idle bridge-attached devices warm, expose stage timing in lifecycle logs, add a create/SSH benchmark script plus Make target, and teach verify.sh that tap-pool-* devices are reusable capacity rather than cleanup leaks. Validated with go test ./..., make build, ./verify.sh, and make bench-create ARGS="--runs 2".
This commit is contained in:
parent
a14a80fd6b
commit
c8d9a122f9
24 changed files with 695 additions and 44 deletions
33
README.md
33
README.md
|
|
@ -26,6 +26,8 @@ The bundle contains:
|
|||
- `bundle.json` with the bundle's default kernel/initrd/modules/rootfs paths
|
||||
- a kernel, initrd, and modules tree referenced by `bundle.json`
|
||||
- `rootfs-docker.ext4`
|
||||
- `rootfs-docker.work-seed.ext4` when present, used to seed `/root` quickly on
|
||||
new VM creates
|
||||
- `rootfs.ext4` when present
|
||||
- `packages.apt`
|
||||
- `id_ed25519`
|
||||
|
|
@ -162,12 +164,14 @@ repo-built `./banger`. You can override either with `runtime_dir` in
|
|||
Useful config keys:
|
||||
- `log_level`
|
||||
- `runtime_dir`
|
||||
- `tap_pool_size`
|
||||
- `firecracker_bin`
|
||||
- `ssh_key_path`
|
||||
- `namegen_path`
|
||||
- `customize_script` (manual helper compatibility; `banger image build` is Go-native)
|
||||
- `vsock_ping_helper_path`
|
||||
- `default_rootfs`
|
||||
- `default_work_seed`
|
||||
- `default_base_rootfs`
|
||||
- `default_kernel`
|
||||
- `default_initrd`
|
||||
|
|
@ -207,7 +211,9 @@ Rebuilt images install a pinned `mise` at `/usr/local/bin/mise`, activate it
|
|||
for bash login and interactive shells, install `opencode` through `mise`,
|
||||
configure `tmux-resurrect` plus `tmux-continuum` for `root` with periodic
|
||||
autosaves and manual-only restore by default, and bake in the
|
||||
`banger-vsock-pingd` systemd service used by the post-SSH reminder path.
|
||||
`banger-vsock-pingd` systemd service used by the post-SSH reminder path. They
|
||||
also emit a `work-seed.ext4` sidecar that lets new VMs clone a prepared `/root`
|
||||
work disk instead of rebuilding it from scratch on every create.
|
||||
|
||||
Show or delete images:
|
||||
```bash
|
||||
|
|
@ -240,6 +246,12 @@ transparent `.vm` lookups on the host.
|
|||
- VMs share a read-only base rootfs image.
|
||||
- Each VM gets its own sparse writable system overlay for `/`.
|
||||
- Each VM gets its own persistent ext4 work disk mounted at `/root`.
|
||||
- When an image has a `work-seed.ext4` sidecar, new VM creates clone that seed
|
||||
and only resize it when needed. Older images still work, but create more
|
||||
slowly because `/root` must be built from scratch.
|
||||
- The daemon can keep a small idle TAP pool warm in the background so VM create
|
||||
does not need to synchronously create a fresh TAP every time. `tap_pool_size`
|
||||
controls the pool depth.
|
||||
|
||||
## Architecture Notes
|
||||
The Go daemon is the primary control plane. VM host integrations such as the
|
||||
|
|
@ -261,6 +273,9 @@ To rebuild the source-checkout default image in `./runtime/rootfs-docker.ext4`:
|
|||
make rootfs
|
||||
```
|
||||
|
||||
That rebuild also regenerates `./runtime/rootfs-docker.work-seed.ext4`, which
|
||||
the daemon uses to speed up future `vm create` calls.
|
||||
|
||||
If your runtime bundle does not include `./runtime/rootfs.ext4`, pass an
|
||||
explicit base image instead:
|
||||
```bash
|
||||
|
|
@ -293,6 +308,22 @@ That writes `dist/banger-runtime.tar.gz` and prints its SHA256 so you can update
|
|||
a local manifest copy before testing bootstrap changes or publishing the
|
||||
archive elsewhere.
|
||||
|
||||
## Benchmarking Create Time
|
||||
Benchmark the current host's `vm create` wall time plus first-SSH readiness:
|
||||
```bash
|
||||
make bench-create
|
||||
```
|
||||
|
||||
Pass options through `ARGS`, for example:
|
||||
```bash
|
||||
make bench-create ARGS="--runs 3 --image docker-dev"
|
||||
```
|
||||
|
||||
The benchmark prints JSON with:
|
||||
- `create_ms`: wall time for `banger vm create`
|
||||
- `ssh_ready_ms`: wall time from create start until `banger vm ssh <vm> -- true`
|
||||
succeeds
|
||||
|
||||
## Remaining Shell Helpers
|
||||
The runtime VM lifecycle is managed through `banger`. The remaining shell scripts are not the primary user interface:
|
||||
- `customize.sh`: manual reference flow for rootfs customization; `banger image build` is now Go-native, but the script still reads
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue