thaloco/aman
1
Fork 0
Code Pull requests Releases Packages Activity Actions

Harden runtime diagnostics for milestone 3

Browse source 
Make the milestone 3 runtime story predictable instead of treating doctor, self-check, and startup failures as loosely related surfaces.

Split doctor and self-check into distinct read-only flows, add tri-state diagnostic status with stable IDs and next steps, and reuse that wording in CLI output, service logs, and tray-triggered diagnostics. Add non-mutating config/model probes, a make runtime-check gate, and public recovery/validation docs for the X11 GA roadmap.

Validation: make runtime-check; PYTHONPATH=src python3 -m unittest discover -s tests -p 'test_*.py'; python3 -m py_compile src/*.py tests/*.py; PYTHONPATH=src python3 -m aman doctor --help; PYTHONPATH=src python3 -m aman self-check --help. Leave milestone 3 open in the roadmap until the manual X11 validation rows are filled.
This commit is contained in:
Thales Maciel 2026-03-12 17:41:23 -03:00
parent a3368056ff
commit ed1b59240b
No known key found for this signature in database
GPG key ID: 33112E6833C34679
16 changed files with 1298 additions and 248 deletions
Download patch file Download diff file Expand all files Collapse all files

44
docs/x11-ga/runtime-validation-report.md Normal file
View file

@ -0,0 +1,44 @@
# Runtime Validation Report
This document tracks milestone 3 evidence for runtime reliability and
diagnostics.
## Automated evidence
Completed on 2026-03-12:
- `PYTHONPATH=src python3 -m unittest tests.test_diagnostics tests.test_aman_cli tests.test_aman tests.test_aiprocess`
- covers `doctor` versus `self-check`, tri-state diagnostic output, warning
versus failure exit codes, read-only model cache probing, and actionable
runtime log wording for audio, hotkey, injection, editor, and startup
failures
- `PYTHONPATH=src python3 -m unittest discover -s tests -p 'test_*.py'`
- confirms the runtime and diagnostics changes do not regress the broader
daemon, CLI, config, and portable bundle flows
- `python3 -m py_compile src/*.py tests/*.py`
- verifies the updated runtime and diagnostics modules compile cleanly
## Automated scenario coverage
| Scenario | Evidence | Status | Notes |
| --- | --- | --- | --- |
| `doctor` and `self-check` have distinct roles | `tests.test_diagnostics`, `tests.test_aman_cli` | Complete | `self-check` extends `doctor` with service/model/startup readiness checks |
| Missing config remains read-only | `tests.test_diagnostics` | Complete | Missing config yields `warn` and does not write a default file |
| Managed model cache probing is read-only | `tests.test_diagnostics`, `tests.test_aiprocess` | Complete | `self-check` uses cache probing and does not download or repair |
| Warning-only diagnostics exit `0`; failures exit `2` | `tests.test_aman_cli` | Complete | Human and JSON output share the same status model |
| Runtime failures log stable IDs and one next step | `tests.test_aman_cli`, `tests.test_aman` | Complete | Covers hotkey, audio-input, injection, editor, and startup failure wording |
| Repeated start/stop and shutdown return to `idle` | `tests.test_aman` | Complete | Current daemon tests cover start, stop, cancel, pause, and shutdown paths |
## Manual X11 validation
These rows must be filled with release-specific evidence before milestone 3 can
be closed as complete for GA signoff.
| Scenario | Debian/Ubuntu | Arch | Fedora | openSUSE | Reviewer | Status | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Service restart after a successful install | Pending | Pending | Pending | Pending | Pending | Pending | Verify `systemctl --user restart aman` returns to the tray/ready state |
| Reboot followed by successful reuse | Pending | Pending | Pending | Pending | Pending | Pending | Validate recovery after a real session restart |
| Offline startup with an already-cached model | Pending | Pending | Pending | Pending | Pending | Pending | Disable network, then confirm the cached path still starts |
| Missing runtime dependency recovery | Pending | Pending | Pending | Pending | Pending | Pending | Remove one documented dependency, verify diagnostics point to the correct fix |
| Tray-triggered diagnostics logging | Pending | Pending | Pending | Pending | Pending | Pending | Use `Run Diagnostics` and confirm the same IDs/messages appear in logs |
| Service-failure escalation path | Pending | Pending | Pending | Pending | Pending | Pending | Confirm `doctor` -> `self-check` -> `journalctl` -> `aman run --verbose` is enough to explain the failure |