Thales Maciel
3cf33d1e0a
Move the default guest package list into a repo manifest and record a hash beside built rootfs images so run/make-rootfs can warn when the docker-ready image is stale. Switch the Firecracker launch path to a single sparse root overlay per VM instead of separate /home and /var disks, so many VMs can share the same base image while still installing packages under /var and working from /root. Keep older images bootable by masking stale home.mount and var.mount units at boot, and scrub those obsolete fstab entries when customize.sh rebuilds an image. Verified with bash -n on the updated scripts; no live VM boot was run in this environment.
149 lines
5 KiB
Markdown
149 lines
5 KiB
Markdown
# banger
|
|
|
|
Minimal Firecracker launcher.
|
|
|
|
## Requirements
|
|
- Linux host with KVM (`/dev/kvm` access)
|
|
- `sudo`, `ip`, `curl`, `ssh`, `jq`
|
|
- `dmsetup`, `losetup`, `blockdev` (device-mapper snapshot for rootfs)
|
|
- `e2cp`, `e2rm` (writes hostname and resolv.conf into rootfs snapshot)
|
|
|
|
## Files
|
|
- `firecracker`: Firecracker binary
|
|
- `wtf/root/boot/vmlinux-6.8.0-94-generic`: guest kernel
|
|
- `wtf/root/boot/initrd.img-6.8.0-94-generic`: guest initrd
|
|
- `wtf/root/lib/modules/6.8.0-94-generic/`: guest kernel modules
|
|
- `rootfs.ext4`: guest root filesystem (base image if present)
|
|
- `rootfs-docker.ext4`: docker-ready guest rootfs (built via `make-rootfs.sh`)
|
|
- `packages.apt`: apt packages baked into rebuilt guest images
|
|
- `id_ed25519`: SSH key for `root`
|
|
- `mapdns`: local DNS mapping CLI used to publish `<vm-name>.vm` → guest IP records
|
|
|
|
## Run
|
|
```
|
|
./run.sh
|
|
```
|
|
|
|
## Run Options
|
|
```
|
|
./run.sh --name calm-otter --vcpu 4 --ram 2048 --overlay-size 12G
|
|
```
|
|
- `--name`: must be unique and match `[a-z0-9][a-z0-9-]{0,63}`.
|
|
- `--vcpu`: defaults to 2, max 16.
|
|
- `--ram`: MiB, defaults to 1024, max 32768.
|
|
- `--overlay-size`: writable dm-snapshot size for VM changes under `/`, including `/root` and `/var` (default: 8G).
|
|
- `--rootfs`: path to the rootfs image (default: `./rootfs-docker.ext4`).
|
|
- `--kernel`: path to the kernel image (default: `./wtf/root/boot/vmlinux-6.8.0-94-generic`).
|
|
- `--initrd`: path to the initrd image (default: `./wtf/root/boot/initrd.img-6.8.0-94-generic`).
|
|
|
|
## Storage Layout
|
|
- `rootfs.ext4` is used as the read-only origin for a per-VM device-mapper snapshot mounted as `/`.
|
|
- Each VM gets one sparse writable overlay file (`cow.ext4`) that stores its changes on top of the shared base image.
|
|
- `/root` and `/var` live inside that per-VM overlay, so VMs can install packages without copying separate disks per VM.
|
|
- `run.sh` masks stale `home.mount` and `var.mount` units at boot so older images with `/dev/vdb` and `/dev/vdc` entries in `/etc/fstab` still boot.
|
|
- `/run` and `/tmp` should be tmpfs via `/etc/fstab`.
|
|
|
|
## SSH
|
|
```
|
|
ssh -i "./id_ed25519" root@<guest_ip>
|
|
```
|
|
|
|
Shortcut:
|
|
```
|
|
./ssh.sh <vm-name-or-ip>
|
|
```
|
|
|
|
## VM DNS
|
|
- Spawned VMs register `<vm-name>.vm` → guest IP through `mapdns set`.
|
|
- VM teardown removes the mapping through `mapdns rm`.
|
|
- `mapdns` writes to `/home/thales/.local/share/mapdns/records.json`.
|
|
|
|
## Internet Access
|
|
VMs do not get internet access by default. You must enable forwarding and NAT:
|
|
```
|
|
./nat.sh up <id-or-name-prefix>
|
|
```
|
|
This enables `net.ipv4.ip_forward=1` and installs per-VM NAT rules for the VM's
|
|
guest IP and TAP device. To remove rules:
|
|
```
|
|
./nat.sh down <id-or-name-prefix>
|
|
```
|
|
Check status with:
|
|
```
|
|
./nat.sh status <id-or-name-prefix>
|
|
```
|
|
|
|
## Shutdown
|
|
```
|
|
reboot
|
|
```
|
|
|
|
## Customize Rootfs (Docker + Kernel Modules)
|
|
Use `customize.sh` to build a writable rootfs with Docker and kernel modules
|
|
preloaded so Docker works out of the box. Pass the base rootfs as a positional
|
|
argument; the output defaults to `docker-<base filename>` in the same directory
|
|
unless you pass `--out`.
|
|
|
|
Base guest packages come from `./packages.apt`. Edit that file to bake tools
|
|
like `vim` and `tmux` into rebuilt images.
|
|
|
|
```
|
|
./customize.sh ./rootfs.ext4 --size 6G --docker
|
|
```
|
|
|
|
Options:
|
|
- `--size`: optional size for the output image.
|
|
- `--kernel`: kernel path (default: `./wtf/root/boot/vmlinux-6.8.0-94-generic`).
|
|
- `--initrd`: initrd path (default: `./wtf/root/boot/initrd.img-6.8.0-94-generic`).
|
|
- `--modules`: kernel modules directory (default: `./wtf/root/lib/modules/6.8.0-94-generic`).
|
|
- `--docker`: install Docker packages into the image.
|
|
- `--out`: output rootfs path (default: `docker-<base filename>`).
|
|
|
|
After boot, enable NAT and validate Docker:
|
|
```
|
|
./nat.sh up <id-or-name-prefix>
|
|
ssh -i "./id_ed25519" root@<guest_ip> "systemctl enable --now docker"
|
|
ssh -i "./id_ed25519" root@<guest_ip> "docker run --rm hello-world"
|
|
```
|
|
|
|
## Build Rootfs On Demand
|
|
`run.sh` defaults to `./rootfs-docker.ext4`. If it is missing, `run.sh` will
|
|
invoke `make-rootfs.sh` to build it.
|
|
|
|
```
|
|
./make-rootfs.sh
|
|
```
|
|
|
|
`make-rootfs.sh` chooses the first available base image:
|
|
- `./rootfs.ext4`
|
|
|
|
If `./packages.apt` changes after `rootfs-docker.ext4` is built, `run.sh` will
|
|
warn and keep using the existing image. `make-rootfs.sh` will also warn and
|
|
exit without rebuilding while the image already exists.
|
|
|
|
To rebuild after package changes:
|
|
```
|
|
rm -f ./rootfs-docker.ext4 ./rootfs-docker.ext4.packages.sha256
|
|
./make-rootfs.sh
|
|
```
|
|
|
|
## Interactive Customization
|
|
To create a writable copy and customize it manually over SSH (no automatic
|
|
package/config changes), use:
|
|
|
|
```
|
|
./interactive.sh ./rootfs-docker.ext4
|
|
```
|
|
|
|
You can override the output path:
|
|
```
|
|
./interactive.sh ./rootfs-docker.ext4 --out ./my-rootfs.ext4
|
|
```
|
|
|
|
## VM Info File
|
|
Each VM writes:
|
|
- `state/vms/<id>/vm.json`: local metadata under `.meta` plus the raw Firecracker config under `.config`.
|
|
|
|
## Log Notes
|
|
- `PCI: Fatal: No config space access function found` and `MissingAddressRange` lines are expected with `pci=off` in `run.sh`.
|
|
- `SELinux: Could not open policy file ...` is expected in the minimal rootfs.
|