Split aman.py into focused CLI and runtime modules
Break the old god module into flat siblings for CLI parsing, run lifecycle, daemon state, shared processing helpers, benchmark tooling, and maintainer-only model sync so changes stop sharing one giant import graph. Keep aman as a thin shim over aman_cli, move sync-default-model behind the hidden aman-maint entrypoint plus Make wrappers, and update packaging metadata plus maintainer docs to reflect the new surface. Retarget the tests to the new seams with dedicated runtime, run, benchmark, maintainer, and entrypoint suites, and verify with python3 -m unittest discover -s tests -p "test_*.py", python3 -m py_compile src/*.py tests/*.py, PYTHONPATH=src python3 -m aman --help, PYTHONPATH=src python3 -m aman version, and PYTHONPATH=src python3 -m aman_maint --help.
This commit is contained in:
parent
721248ca26
commit
4d0081d1d0
18 changed files with 2838 additions and 2427 deletions
|
|
@ -67,7 +67,6 @@ aman run --config ~/.config/aman/config.json
|
|||
aman bench --text "example transcript" --repeat 5 --warmup 1
|
||||
aman build-heuristic-dataset --input benchmarks/heuristics_dataset.raw.jsonl --output benchmarks/heuristics_dataset.jsonl --json
|
||||
aman eval-models --dataset benchmarks/cleanup_dataset.jsonl --matrix benchmarks/model_matrix.small_first.json --heuristic-dataset benchmarks/heuristics_dataset.jsonl --heuristic-weight 0.25 --json
|
||||
aman sync-default-model --check --report benchmarks/results/latest.json --artifacts benchmarks/model_artifacts.json --constants src/constants.py
|
||||
aman version
|
||||
aman init --config ~/.config/aman/config.json --force
|
||||
```
|
||||
|
|
@ -88,14 +87,20 @@ alignment/editor/fact-guard/vocabulary cleanup and prints timing summaries.
|
|||
```bash
|
||||
aman build-heuristic-dataset --input benchmarks/heuristics_dataset.raw.jsonl --output benchmarks/heuristics_dataset.jsonl
|
||||
aman eval-models --dataset benchmarks/cleanup_dataset.jsonl --matrix benchmarks/model_matrix.small_first.json --heuristic-dataset benchmarks/heuristics_dataset.jsonl --heuristic-weight 0.25 --output benchmarks/results/latest.json
|
||||
aman sync-default-model --report benchmarks/results/latest.json --artifacts benchmarks/model_artifacts.json --constants src/constants.py
|
||||
make sync-default-model
|
||||
```
|
||||
|
||||
- `eval-models` runs a structured model/parameter sweep over a JSONL dataset
|
||||
and outputs latency plus quality metrics.
|
||||
- When `--heuristic-dataset` is provided, the report also includes
|
||||
alignment-heuristic quality metrics.
|
||||
- `sync-default-model` promotes the report winner to the managed default model
|
||||
constants and can be run in `--check` mode for CI and release gates.
|
||||
- `make sync-default-model` promotes the report winner to the managed default
|
||||
model constants and `make check-default-model` keeps that drift check in CI.
|
||||
|
||||
Internal maintainer CLI:
|
||||
|
||||
```bash
|
||||
aman-maint sync-default-model --check --report benchmarks/results/latest.json --artifacts benchmarks/model_artifacts.json --constants src/constants.py
|
||||
```
|
||||
|
||||
Dataset and artifact details live in [`benchmarks/README.md`](../benchmarks/README.md).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue