Thales Maciel
f779b71e1b
Stop letting the explicit compile step overstate its coverage. The old py_compile globs only touched top-level modules, so syntax errors in nested packages could slip past make check and release-check.\n\nAdd a shared compile-check recipe in Makefile that runs python -m compileall -q src tests, and have both check and release-check use it so the local verification paths stay aligned. Update the GitHub Actions compile step and the matching runtime validation evidence doc to describe the same recursive compile contract.\n\nValidate with python3 -m compileall -q src tests, make check, and make release-check.
3.9 KiB
3.9 KiB
Runtime Validation Report
This document tracks milestone 3 evidence for runtime reliability and diagnostics.
Automated evidence
Completed on 2026-03-12:
PYTHONPATH=src python3 -m unittest tests.test_diagnostics tests.test_aman_cli tests.test_aman tests.test_aiprocess- covers
doctorversusself-check, tri-state diagnostic output, warning versus failure exit codes, read-only model cache probing, and actionable runtime log wording for audio, hotkey, injection, editor, and startup failures
- covers
PYTHONPATH=src python3 -m unittest discover -s tests -p 'test_*.py'- confirms the runtime and diagnostics changes do not regress the broader daemon, CLI, config, and portable bundle flows
python3 -m compileall -q src tests- verifies the updated runtime, diagnostics, and nested package modules compile cleanly
Automated scenario coverage
| Scenario | Evidence | Status | Notes |
|---|---|---|---|
doctor and self-check have distinct roles |
tests.test_diagnostics, tests.test_aman_cli |
Complete | self-check extends doctor with service/model/startup readiness checks |
| Missing config remains read-only | tests.test_diagnostics |
Complete | Missing config yields warn and does not write a default file |
| Managed model cache probing is read-only | tests.test_diagnostics, tests.test_aiprocess |
Complete | self-check uses cache probing and does not download or repair |
Warning-only diagnostics exit 0; failures exit 2 |
tests.test_aman_cli |
Complete | Human and JSON output share the same status model |
| Runtime failures log stable IDs and one next step | tests.test_aman_cli, tests.test_aman |
Complete | Covers hotkey, audio-input, injection, editor, and startup failure wording |
Repeated start/stop and shutdown return to idle |
tests.test_aman |
Complete | Current daemon tests cover start, stop, cancel, pause, and shutdown paths |
Manual X11 validation
One representative distro family with real runtime validation is enough to close milestone 3 for now. Full Debian/Ubuntu, Arch, Fedora, and openSUSE coverage remains a milestone 5 GA signoff requirement.
Store raw evidence for each runtime validation pass in
user-readiness/<linux-timestamp>.md and reference that file in the Notes
column.
| Scenario | Debian/Ubuntu | Arch | Fedora | openSUSE | Reviewer | Status | Notes |
|---|---|---|---|---|---|---|---|
| Service restart after a successful install | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; verify systemctl --user restart aman returns to the tray/ready state |
| Reboot followed by successful reuse | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; validate recovery after a real session restart |
| Offline startup with an already-cached model | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; cached-model offline start succeeded |
| Missing runtime dependency recovery | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; diagnostics pointed to the fix |
| Tray-triggered diagnostics logging | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; Run Diagnostics matched the documented log path |
| Service-failure escalation path | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in 1773357669.md; doctor -> self-check -> journalctl -> aman run --verbose was sufficient |