Thales Maciel
b4a3d446fa
Some checks failed
ci / test-and-build (push) Has been cancelled
Record the user-reported Arch X11 validation pass and thread it through the portable and runtime validation matrices. Adjust the milestone 2 and 3 closeout wording so one fully validated representative distro family is enough for now, while keeping Debian/Ubuntu, Fedora, and openSUSE coverage as an explicit milestone 5 GA signoff requirement. Update the roadmap and GA validation rollup to mark milestones 2 and 3 complete for now rather than fully GA-complete, and archive the raw Arch evidence in user-readiness/1773357669.md. Validation: documentation consistency review only; no code or behavior changes were made.
49 lines
3.8 KiB
Markdown
49 lines
3.8 KiB
Markdown
# Runtime Validation Report
|
|
|
|
This document tracks milestone 3 evidence for runtime reliability and
|
|
diagnostics.
|
|
|
|
## Automated evidence
|
|
|
|
Completed on 2026-03-12:
|
|
|
|
- `PYTHONPATH=src python3 -m unittest tests.test_diagnostics tests.test_aman_cli tests.test_aman tests.test_aiprocess`
|
|
- covers `doctor` versus `self-check`, tri-state diagnostic output, warning
|
|
versus failure exit codes, read-only model cache probing, and actionable
|
|
runtime log wording for audio, hotkey, injection, editor, and startup
|
|
failures
|
|
- `PYTHONPATH=src python3 -m unittest discover -s tests -p 'test_*.py'`
|
|
- confirms the runtime and diagnostics changes do not regress the broader
|
|
daemon, CLI, config, and portable bundle flows
|
|
- `python3 -m py_compile src/*.py tests/*.py`
|
|
- verifies the updated runtime and diagnostics modules compile cleanly
|
|
|
|
## Automated scenario coverage
|
|
|
|
| Scenario | Evidence | Status | Notes |
|
|
| --- | --- | --- | --- |
|
|
| `doctor` and `self-check` have distinct roles | `tests.test_diagnostics`, `tests.test_aman_cli` | Complete | `self-check` extends `doctor` with service/model/startup readiness checks |
|
|
| Missing config remains read-only | `tests.test_diagnostics` | Complete | Missing config yields `warn` and does not write a default file |
|
|
| Managed model cache probing is read-only | `tests.test_diagnostics`, `tests.test_aiprocess` | Complete | `self-check` uses cache probing and does not download or repair |
|
|
| Warning-only diagnostics exit `0`; failures exit `2` | `tests.test_aman_cli` | Complete | Human and JSON output share the same status model |
|
|
| Runtime failures log stable IDs and one next step | `tests.test_aman_cli`, `tests.test_aman` | Complete | Covers hotkey, audio-input, injection, editor, and startup failure wording |
|
|
| Repeated start/stop and shutdown return to `idle` | `tests.test_aman` | Complete | Current daemon tests cover start, stop, cancel, pause, and shutdown paths |
|
|
|
|
## Manual X11 validation
|
|
|
|
One representative distro family with real runtime validation is enough to
|
|
close milestone 3 for now. Full Debian/Ubuntu, Arch, Fedora, and openSUSE
|
|
coverage remains a milestone 5 GA signoff requirement.
|
|
|
|
Store raw evidence for each runtime validation pass in
|
|
`user-readiness/<linux-timestamp>.md` and reference that file in the `Notes`
|
|
column.
|
|
|
|
| Scenario | Debian/Ubuntu | Arch | Fedora | openSUSE | Reviewer | Status | Notes |
|
|
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
| Service restart after a successful install | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); verify `systemctl --user restart aman` returns to the tray/ready state |
|
|
| Reboot followed by successful reuse | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); validate recovery after a real session restart |
|
|
| Offline startup with an already-cached model | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); cached-model offline start succeeded |
|
|
| Missing runtime dependency recovery | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); diagnostics pointed to the fix |
|
|
| Tray-triggered diagnostics logging | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); `Run Diagnostics` matched the documented log path |
|
|
| Service-failure escalation path | Pending | Pass | Pending | Pending | User | Arch validated | User-reported Arch X11 validation in [`1773357669.md`](../../user-readiness/1773357669.md); `doctor` -> `self-check` -> `journalctl` -> `aman run --verbose` was sufficient |
|