diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d204a3..27b4d47 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,17 @@ All notable user-visible changes to `pyro-mcp` are documented here. +## 3.1.0 + +- Added explicit workspace lifecycle stop/start operations across the CLI, Python SDK, and MCP + server so a persistent workspace can be paused and resumed without resetting `/workspace`, + snapshots, or command history. +- Added secondary stopped-workspace disk tools with raw ext4 export plus offline `disk list` and + `disk read` inspection for guest-backed workspaces. +- Scrubbed guest runtime-only paths such as `/run/pyro-secrets`, `/run/pyro-shells`, and + `/run/pyro-services` before stopped-workspace disk export and offline inspection so those tools + stay secondary to the stable workspace product without leaking runtime-only state. + ## 3.0.0 - Promoted the workspace-first product surface to stable across the CLI, Python SDK, and MCP diff --git a/README.md b/README.md index b0886c1..674dedd 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ It exposes the same runtime in three public forms: - Stable workspace walkthrough GIF: [docs/assets/workspace-first-run.gif](docs/assets/workspace-first-run.gif) - Terminal walkthrough GIF: [docs/assets/first-run.gif](docs/assets/first-run.gif) - PyPI package: [pypi.org/project/pyro-mcp](https://pypi.org/project/pyro-mcp/) -- What's new in 3.0.0: [CHANGELOG.md#300](CHANGELOG.md#300) +- What's new in 3.1.0: [CHANGELOG.md#310](CHANGELOG.md#310) - Host requirements: [docs/host-requirements.md](docs/host-requirements.md) - Integration targets: [docs/integrations.md](docs/integrations.md) - Public contract: [docs/public-contract.md](docs/public-contract.md) @@ -58,7 +58,7 @@ What success looks like: ```bash Platform: linux-x86_64 Runtime: PASS -Catalog version: 3.0.0 +Catalog version: 3.1.0 ... [pull] phase=install environment=debian:12 [pull] phase=ready environment=debian:12 @@ -107,6 +107,7 @@ That stable workspace path gives you: - full-sandbox recovery with `workspace reset` - baseline comparison with `workspace diff` - explicit host-out export with `workspace export` +- secondary stopped-workspace disk inspection with `workspace stop|start` and `workspace disk *` After the quickstart works: @@ -123,6 +124,8 @@ After the quickstart works: - open a persistent interactive shell with `uvx --from pyro-mcp pyro workspace shell open WORKSPACE_ID` - start long-running workspace services with `uvx --from pyro-mcp pyro workspace service start WORKSPACE_ID app --ready-file .ready -- sh -lc 'touch .ready && while true; do sleep 60; done'` - publish one guest service port to the host with `uvx --from pyro-mcp pyro workspace create debian:12 --network-policy egress+published-ports` and `uvx --from pyro-mcp pyro workspace service start WORKSPACE_ID app --ready-http http://127.0.0.1:8080/ --publish 18080:8080 -- ./start-app` +- stop a workspace for offline inspection with `uvx --from pyro-mcp pyro workspace stop WORKSPACE_ID` +- inspect or export one stopped guest rootfs with `uvx --from pyro-mcp pyro workspace disk list WORKSPACE_ID`, `uvx --from pyro-mcp pyro workspace disk read WORKSPACE_ID note.txt`, and `uvx --from pyro-mcp pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4` - move to Python or MCP via [docs/integrations.md](docs/integrations.md) ## Supported Hosts @@ -176,7 +179,7 @@ uvx --from pyro-mcp pyro env list Expected output: ```bash -Catalog version: 3.0.0 +Catalog version: 3.1.0 debian:12 [installed|not installed] Debian 12 environment with Git preinstalled for common agent workflows. debian:12-base [installed|not installed] Minimal Debian 12 environment for shell and core Unix tooling. debian:12-build [installed|not installed] Debian 12 environment with Git and common build tools preinstalled. @@ -275,6 +278,11 @@ pyro workspace service status WORKSPACE_ID web pyro workspace service logs WORKSPACE_ID web --tail-lines 50 pyro workspace service stop WORKSPACE_ID web pyro workspace service stop WORKSPACE_ID worker +pyro workspace stop WORKSPACE_ID +pyro workspace disk list WORKSPACE_ID +pyro workspace disk read WORKSPACE_ID src/note.txt +pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4 +pyro workspace start WORKSPACE_ID pyro workspace logs WORKSPACE_ID pyro workspace delete WORKSPACE_ID ``` @@ -283,7 +291,7 @@ Persistent workspaces start in `/workspace` and keep command history until you d machine consumption, add `--json` and read the returned `workspace_id`. Use `--seed-path` when you want the workspace to start from a host directory or a local `.tar` / `.tar.gz` / `.tgz` archive instead of an empty workspace. Use `pyro workspace sync push` when you want to import -later host-side changes into a started workspace. Sync is non-atomic in `3.0.0`; if it fails +later host-side changes into a started workspace. Sync is non-atomic in `3.1.0`; if it fails partway through, prefer `pyro workspace reset` to recover from `baseline` or one named snapshot. Use `pyro workspace diff` to compare the live `/workspace` tree to its immutable create-time baseline, and `pyro workspace export` to copy one changed file or directory back to the host. Use @@ -301,7 +309,9 @@ service must be probed from the host on `127.0.0.1`. Use `--secret` and `--secret-file` at workspace creation when the sandbox needs private tokens or config. Persisted secrets are materialized inside the guest at `/run/pyro-secrets/`, and `--secret-env SECRET_NAME[=ENV_VAR]` maps one secret into one exec, shell, or service call without -exposing the raw value in workspace status, logs, diffs, or exports. +exposing the raw value in workspace status, logs, diffs, or exports. Use `pyro workspace stop` +plus `pyro workspace disk list|read|export` when you need offline inspection or one raw ext4 copy +from a stopped guest-backed workspace, then `pyro workspace start` to resume the same workspace. ## Public Interfaces diff --git a/docs/first-run.md b/docs/first-run.md index cdd5be3..82216f3 100644 --- a/docs/first-run.md +++ b/docs/first-run.md @@ -22,7 +22,7 @@ Networking: tun=yes ip_forward=yes ```bash $ uvx --from pyro-mcp pyro env list -Catalog version: 3.0.0 +Catalog version: 3.1.0 debian:12 [installed|not installed] Debian 12 environment with Git preinstalled for common agent workflows. debian:12-base [installed|not installed] Minimal Debian 12 environment for shell and core Unix tooling. debian:12-build [installed|not installed] Debian 12 environment with Git and common build tools preinstalled. @@ -81,6 +81,11 @@ $ uvx --from pyro-mcp pyro workspace snapshot create "$WORKSPACE_ID" checkpoint $ uvx --from pyro-mcp pyro workspace service start "$WORKSPACE_ID" web --ready-file .web-ready -- sh -lc 'touch .web-ready && while true; do sleep 60; done' $ uvx --from pyro-mcp pyro workspace reset "$WORKSPACE_ID" --snapshot checkpoint $ uvx --from pyro-mcp pyro workspace export "$WORKSPACE_ID" note.txt --output ./note.txt +$ uvx --from pyro-mcp pyro workspace stop "$WORKSPACE_ID" +$ uvx --from pyro-mcp pyro workspace disk list "$WORKSPACE_ID" +$ uvx --from pyro-mcp pyro workspace disk read "$WORKSPACE_ID" note.txt +$ uvx --from pyro-mcp pyro workspace disk export "$WORKSPACE_ID" --output ./workspace.ext4 +$ uvx --from pyro-mcp pyro workspace start "$WORKSPACE_ID" $ uvx --from pyro-mcp pyro workspace delete "$WORKSPACE_ID" ``` @@ -200,12 +205,33 @@ $ uvx --from pyro-mcp pyro workspace service stop WORKSPACE_ID web $ uvx --from pyro-mcp pyro workspace service stop WORKSPACE_ID worker [workspace-service-stop] workspace_id=... service=worker state=stopped execution_mode=guest_vsock + +$ uvx --from pyro-mcp pyro workspace stop WORKSPACE_ID +Workspace ID: ... +State: stopped + +$ uvx --from pyro-mcp pyro workspace disk list WORKSPACE_ID src --recursive +Workspace: ... +Path: /workspace/src +- /workspace/src [directory] +- /workspace/src/note.txt [file] bytes=... + +$ uvx --from pyro-mcp pyro workspace disk read WORKSPACE_ID src/note.txt +hello from synced workspace +[workspace-disk-read] workspace_id=... path=/workspace/src/note.txt size_bytes=... truncated=False + +$ uvx --from pyro-mcp pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4 +[workspace-disk-export] workspace_id=... output_path=... disk_format=ext4 bytes_written=... + +$ uvx --from pyro-mcp pyro workspace start WORKSPACE_ID +Workspace ID: ... +State: started ``` Use `--seed-path` when the workspace should start from a host directory or a local `.tar` / `.tar.gz` / `.tgz` archive instead of an empty `/workspace`. Use `pyro workspace sync push` when you need to import later host-side changes into a started -workspace. Sync is non-atomic in `3.0.0`; if it fails partway through, prefer `pyro workspace reset` +workspace. Sync is non-atomic in `3.1.0`; if it fails partway through, prefer `pyro workspace reset` to recover from `baseline` or one named snapshot. Use `pyro workspace diff` to compare the current `/workspace` tree to its immutable create-time baseline, `pyro workspace snapshot *` to create named checkpoints, and `pyro workspace export` to copy one changed file or directory back to the @@ -219,7 +245,9 @@ service must be reachable from the host on `127.0.0.1`. Use `--secret` and `--se workspace creation when the sandbox needs private tokens or config. Persisted secret files are materialized at `/run/pyro-secrets/`, and `--secret-env SECRET_NAME[=ENV_VAR]` maps one secret into one exec, shell, or service call without storing that environment mapping on the -workspace itself. +workspace itself. Use `pyro workspace stop` plus `pyro workspace disk list|read|export` when you +need offline inspection or one raw ext4 copy from a stopped guest-backed workspace, then +`pyro workspace start` to resume the same workspace. The stable workspace walkthrough GIF in the README is rendered from [docs/assets/workspace-first-run.tape](assets/workspace-first-run.tape) with diff --git a/docs/install.md b/docs/install.md index 9e1e1d5..092fcc4 100644 --- a/docs/install.md +++ b/docs/install.md @@ -85,7 +85,7 @@ uvx --from pyro-mcp pyro env list Expected output: ```bash -Catalog version: 3.0.0 +Catalog version: 3.1.0 debian:12 [installed|not installed] Debian 12 environment with Git preinstalled for common agent workflows. debian:12-base [installed|not installed] Minimal Debian 12 environment for shell and core Unix tooling. debian:12-build [installed|not installed] Debian 12 environment with Git and common build tools preinstalled. @@ -159,6 +159,7 @@ This is the stable persistent-workspace contract: - `workspace snapshot *` and `workspace reset` make reset-over-repair explicit - `workspace diff` compares against the immutable create-time baseline - `workspace export` copies results back to the host +- `workspace stop|start` and `workspace disk *` add secondary stopped-workspace inspection and raw ext4 export ## 6. Optional demo proof point @@ -210,6 +211,7 @@ After the CLI path works, you can move on to: - baseline diff: `pyro workspace diff WORKSPACE_ID` - snapshots and reset: `pyro workspace snapshot create WORKSPACE_ID checkpoint` and `pyro workspace reset WORKSPACE_ID --snapshot checkpoint` - host export: `pyro workspace export WORKSPACE_ID note.txt --output ./note.txt` +- stopped-workspace inspection: `pyro workspace stop WORKSPACE_ID`, `pyro workspace disk list WORKSPACE_ID`, `pyro workspace disk read WORKSPACE_ID note.txt`, and `pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4` - interactive shells: `pyro workspace shell open WORKSPACE_ID` - long-running services: `pyro workspace service start WORKSPACE_ID app --ready-file .ready -- sh -lc 'touch .ready && while true; do sleep 60; done'` - localhost-published ports: `pyro workspace create debian:12 --network-policy egress+published-ports` and `pyro workspace service start WORKSPACE_ID app --ready-http http://127.0.0.1:8080/ --publish 18080:8080 -- ./start-app` @@ -246,6 +248,11 @@ pyro workspace service status WORKSPACE_ID web pyro workspace service logs WORKSPACE_ID web --tail-lines 50 pyro workspace service stop WORKSPACE_ID web pyro workspace service stop WORKSPACE_ID worker +pyro workspace stop WORKSPACE_ID +pyro workspace disk list WORKSPACE_ID +pyro workspace disk read WORKSPACE_ID src/note.txt +pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4 +pyro workspace start WORKSPACE_ID pyro workspace logs WORKSPACE_ID pyro workspace delete WORKSPACE_ID ``` @@ -254,7 +261,7 @@ Workspace commands default to the persistent `/workspace` directory inside the g the identifier programmatically, use `--json` and read the `workspace_id` field. Use `--seed-path` when the workspace should start from a host directory or a local `.tar` / `.tar.gz` / `.tgz` archive. Use `pyro workspace sync push` for later host-side changes to a started workspace. Sync -is non-atomic in `3.0.0`; if it fails partway through, prefer `pyro workspace reset` to recover +is non-atomic in `3.1.0`; if it fails partway through, prefer `pyro workspace reset` to recover from `baseline` or one named snapshot. Use `pyro workspace diff` to compare the current workspace tree to its immutable create-time baseline, `pyro workspace snapshot *` to capture named checkpoints, and `pyro workspace export` to copy one changed file or directory back to the host. Use @@ -268,7 +275,9 @@ service must be reachable from the host on `127.0.0.1`. Use `--secret` and `--se workspace creation when the sandbox needs private tokens or config, and `--secret-env SECRET_NAME[=ENV_VAR]` when one exec, shell, or service call needs that secret as an environment variable. Persisted secret files are available in the guest at -`/run/pyro-secrets/`. +`/run/pyro-secrets/`. Use `pyro workspace stop` plus `pyro workspace disk list|read|export` +when you need offline inspection or one raw ext4 copy from a stopped guest-backed workspace, then +`pyro workspace start` to resume it. ## Contributor Clone diff --git a/docs/integrations.md b/docs/integrations.md index ecfd18f..ffaa2a3 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -35,6 +35,7 @@ Recommended surface: - `workspace_create(..., secrets=...)` + `workspace_exec(..., secret_env=...)` when the workspace needs private tokens or authenticated setup - `workspace_create(..., network_policy="egress+published-ports")` + `start_service(..., published_ports=[...])` when the host must probe one workspace service - `workspace_diff` + `workspace_export` when the agent needs explicit baseline comparison or host-out file transfer +- `stop_workspace(...)` + `list_workspace_disk(...)` / `read_workspace_disk(...)` / `export_workspace_disk(...)` when one stopped guest-backed workspace needs offline inspection or a raw ext4 copy - `start_service` / `list_services` / `status_service` / `logs_service` / `stop_service` when the agent needs long-running processes inside that workspace - `open_shell(..., secret_env=...)` / `read_shell` / `write_shell` when the agent needs an interactive PTY inside that workspace @@ -95,6 +96,9 @@ Lifecycle note: - use `diff_workspace(...)` when the agent needs a structured comparison against the immutable create-time baseline - use `export_workspace(...)` when the agent needs one file or directory copied back to the host +- use `stop_workspace(...)` plus `list_workspace_disk(...)`, `read_workspace_disk(...)`, or + `export_workspace_disk(...)` when the agent needs offline inspection or one raw ext4 copy from + a stopped guest-backed workspace - use `start_service(...)` when the agent needs long-running processes and typed readiness inside one workspace - use `open_shell(...)` when the agent needs interactive shell state instead of one-shot execs diff --git a/docs/public-contract.md b/docs/public-contract.md index 3d70ab4..e5f5fca 100644 --- a/docs/public-contract.md +++ b/docs/public-contract.md @@ -26,8 +26,13 @@ Top-level commands: - `pyro run` - `pyro workspace create` - `pyro workspace sync push` +- `pyro workspace stop` +- `pyro workspace start` - `pyro workspace exec` - `pyro workspace export` +- `pyro workspace disk export` +- `pyro workspace disk list` +- `pyro workspace disk read` - `pyro workspace diff` - `pyro workspace snapshot create` - `pyro workspace snapshot list` @@ -72,7 +77,13 @@ Behavioral guarantees: - `pyro workspace create --network-policy {off,egress,egress+published-ports}` controls workspace guest networking and whether services may publish localhost ports. - `pyro workspace create --secret NAME=VALUE` and `--secret-file NAME=PATH` persist guest-only UTF-8 secrets outside `/workspace`. - `pyro workspace sync push WORKSPACE_ID SOURCE_PATH [--dest WORKSPACE_PATH]` imports later host-side directory or archive content into a started workspace. +- `pyro workspace stop WORKSPACE_ID` stops one persistent workspace without deleting its `/workspace`, snapshots, or command history. +- `pyro workspace start WORKSPACE_ID` restarts one stopped workspace without resetting `/workspace`. - `pyro workspace export WORKSPACE_ID PATH --output HOST_PATH` exports one file or directory from `/workspace` back to the host. +- `pyro workspace disk export WORKSPACE_ID --output HOST_PATH` copies the stopped guest-backed workspace rootfs as raw ext4 to the host. +- `pyro workspace disk list WORKSPACE_ID [PATH] [--recursive]` inspects a stopped guest-backed workspace rootfs offline without booting the guest. +- `pyro workspace disk read WORKSPACE_ID PATH [--max-bytes N]` reads one regular file from a stopped guest-backed workspace rootfs offline. +- `pyro workspace disk *` requires `state=stopped` and a guest-backed workspace; it fails on `host_compat`. - `pyro workspace diff WORKSPACE_ID` compares the current `/workspace` tree to the immutable create-time baseline. - `pyro workspace snapshot *` manages explicit named snapshots in addition to the implicit `baseline`. - `pyro workspace reset WORKSPACE_ID [--snapshot SNAPSHOT_NAME|baseline]` recreates the full sandbox and restores `/workspace` from the chosen snapshot. @@ -108,7 +119,12 @@ Supported public entrypoints: - `Pyro.create_vm(...)` - `Pyro.create_workspace(..., network_policy="off", secrets=None)` - `Pyro.push_workspace_sync(workspace_id, source_path, *, dest="/workspace")` +- `Pyro.stop_workspace(workspace_id)` +- `Pyro.start_workspace(workspace_id)` - `Pyro.export_workspace(workspace_id, path, *, output_path)` +- `Pyro.export_workspace_disk(workspace_id, *, output_path)` +- `Pyro.list_workspace_disk(workspace_id, path="/workspace", recursive=False)` +- `Pyro.read_workspace_disk(workspace_id, path, *, max_bytes=65536)` - `Pyro.diff_workspace(workspace_id)` - `Pyro.create_snapshot(workspace_id, snapshot_name)` - `Pyro.list_snapshots(workspace_id)` @@ -147,7 +163,12 @@ Stable public method names: - `create_vm(...)` - `create_workspace(..., network_policy="off", secrets=None)` - `push_workspace_sync(workspace_id, source_path, *, dest="/workspace")` +- `stop_workspace(workspace_id)` +- `start_workspace(workspace_id)` - `export_workspace(workspace_id, path, *, output_path)` +- `export_workspace_disk(workspace_id, *, output_path)` +- `list_workspace_disk(workspace_id, path="/workspace", recursive=False)` +- `read_workspace_disk(workspace_id, path, *, max_bytes=65536)` - `diff_workspace(workspace_id)` - `create_snapshot(workspace_id, snapshot_name)` - `list_snapshots(workspace_id)` @@ -186,7 +207,13 @@ Behavioral defaults: - `Pyro.create_workspace(..., network_policy="off"|"egress"|"egress+published-ports")` controls workspace guest networking and whether services may publish host ports. - `Pyro.create_workspace(..., secrets=...)` persists guest-only UTF-8 secrets outside `/workspace`. - `Pyro.push_workspace_sync(...)` imports later host-side directory or archive content into a started workspace. +- `Pyro.stop_workspace(...)` stops one persistent workspace without deleting its `/workspace`, snapshots, or command history. +- `Pyro.start_workspace(...)` restarts one stopped workspace without resetting `/workspace`. - `Pyro.export_workspace(...)` exports one file or directory from `/workspace` to an explicit host path. +- `Pyro.export_workspace_disk(...)` copies the stopped guest-backed workspace rootfs as raw ext4 to an explicit host path. +- `Pyro.list_workspace_disk(...)` inspects a stopped guest-backed workspace rootfs offline without booting the guest. +- `Pyro.read_workspace_disk(...)` reads one regular file from a stopped guest-backed workspace rootfs offline. +- stopped-workspace disk helpers require `state=stopped` and a guest-backed workspace; they fail on `host_compat`. - `Pyro.diff_workspace(...)` compares the current `/workspace` tree to the immutable create-time baseline. - `Pyro.create_snapshot(...)` captures one named `/workspace` checkpoint. - `Pyro.list_snapshots(...)` lists the implicit `baseline` plus any named snapshots. @@ -226,8 +253,13 @@ Persistent workspace tools: - `workspace_create` - `workspace_sync_push` +- `workspace_stop` +- `workspace_start` - `workspace_exec` - `workspace_export` +- `workspace_disk_export` +- `workspace_disk_list` +- `workspace_disk_read` - `workspace_diff` - `snapshot_create` - `snapshot_list` @@ -257,7 +289,13 @@ Behavioral defaults: - `workspace_create` accepts `network_policy` with `off`, `egress`, or `egress+published-ports` to control workspace guest networking and service port publication. - `workspace_create` accepts optional `secrets` and persists guest-only UTF-8 secret material outside `/workspace`. - `workspace_sync_push` imports later host-side directory or archive content into a started workspace, with an optional `dest` under `/workspace`. +- `workspace_stop` stops one persistent workspace without deleting its `/workspace`, snapshots, or command history. +- `workspace_start` restarts one stopped workspace without resetting `/workspace`. - `workspace_export` exports one file or directory from `/workspace` to an explicit host path. +- `workspace_disk_export` copies the stopped guest-backed workspace rootfs as raw ext4 to an explicit host path. +- `workspace_disk_list` inspects a stopped guest-backed workspace rootfs offline without booting the guest. +- `workspace_disk_read` reads one regular file from a stopped guest-backed workspace rootfs offline. +- stopped-workspace disk tools require `state=stopped` and a guest-backed workspace; they fail on `host_compat`. - `workspace_diff` compares the current `/workspace` tree to the immutable create-time baseline. - `snapshot_create`, `snapshot_list`, and `snapshot_delete` manage explicit named snapshots in addition to the implicit `baseline`. - `workspace_reset` recreates the full sandbox and restores `/workspace` from `baseline` or one named snapshot. diff --git a/docs/roadmap/task-workspace-ga.md b/docs/roadmap/task-workspace-ga.md index d824f4a..b5482b4 100644 --- a/docs/roadmap/task-workspace-ga.md +++ b/docs/roadmap/task-workspace-ga.md @@ -2,7 +2,7 @@ This roadmap turns the agent-workspace vision into release-sized milestones. -Current baseline is `3.0.0`: +Current baseline is `3.1.0`: - workspace persistence exists and the public surface is now workspace-first - host crossing currently covers create-time seeding, later sync push, and explicit export @@ -38,12 +38,12 @@ also expected to update: 6. [`2.9.0` Secrets](task-workspace-ga/2.9.0-secrets.md) - Done 7. [`2.10.0` Network Policy And Host Port Publication](task-workspace-ga/2.10.0-network-policy-and-host-port-publication.md) - Done 8. [`3.0.0` Stable Workspace Product](task-workspace-ga/3.0.0-stable-workspace-product.md) - Done -9. [`3.1.0` Secondary Disk Tools](task-workspace-ga/3.1.0-secondary-disk-tools.md) +9. [`3.1.0` Secondary Disk Tools](task-workspace-ga/3.1.0-secondary-disk-tools.md) - Done -## Remaining Follow-Up +## Roadmap Status -The core workspace product is now stable. The remaining planned follow-up is intentionally -secondary: +The planned workspace roadmap is complete. -- `3.1.0` secondary disk tools for offline inspection and disk-level workflows -- no further roadmap milestone changes the stable workspace-first core contract +- `3.1.0` added secondary stopped-workspace disk export and offline inspection helpers without + changing the stable workspace-first core contract. +- Future work, if any, is now outside the planned vision milestones tracked in this roadmap. diff --git a/docs/roadmap/task-workspace-ga/3.1.0-secondary-disk-tools.md b/docs/roadmap/task-workspace-ga/3.1.0-secondary-disk-tools.md index a9015db..593d71e 100644 --- a/docs/roadmap/task-workspace-ga/3.1.0-secondary-disk-tools.md +++ b/docs/roadmap/task-workspace-ga/3.1.0-secondary-disk-tools.md @@ -1,38 +1,59 @@ # `3.1.0` Secondary Disk Tools +Status: Done + ## Goal -Add the disk-level tools the vision explicitly places last, while keeping them -secondary to the workspace identity. +Add stopped-workspace disk tools the vision explicitly places last, while keeping them secondary +to the stable workspace identity. ## Public API Changes -Representative additions: +Shipped additions: -- stopped-workspace disk export/import helpers -- offline inspection helpers -- disk-oriented snapshot inspection - -Exact command names should reinforce that these are supporting tools rather than -the primary product contract. +- `pyro workspace stop WORKSPACE_ID` +- `pyro workspace start WORKSPACE_ID` +- `pyro workspace disk export WORKSPACE_ID --output HOST_PATH` +- `pyro workspace disk list WORKSPACE_ID [PATH] [--recursive]` +- `pyro workspace disk read WORKSPACE_ID PATH [--max-bytes N]` +- matching Python SDK methods: + - `stop_workspace` + - `start_workspace` + - `export_workspace_disk` + - `list_workspace_disk` + - `read_workspace_disk` +- matching MCP tools: + - `workspace_stop` + - `workspace_start` + - `workspace_disk_export` + - `workspace_disk_list` + - `workspace_disk_read` ## Implementation Boundaries -- keep these tools scoped to seeding, inspection, and offline workflows +- keep these tools scoped to stopped-workspace inspection, export, and offline workflows - do not replace shell, exec, services, diff, export, or reset as the main interaction model - prefer explicit stopped-workspace or offline semantics +- require guest-backed workspaces for `workspace disk *` +- keep disk export raw ext4 only in this milestone +- scrub runtime-only guest paths such as `/run/pyro-secrets`, `/run/pyro-shells`, and + `/run/pyro-services` before offline inspection or export ## Non-Goals - no drift into generic image tooling identity - no replacement of workspace-level host crossing +- no disk import +- no disk mutation +- no create-from-disk workflow ## Acceptance Scenarios - inspect or export a stopped workspace disk for offline analysis -- import or snapshot content through disk-level tools without changing the main - workspace workflow +- stop a workspace, inspect `/workspace` offline, export raw ext4, then start the same workspace + again without resetting `/workspace` +- verify secret-backed workspaces scrub runtime-only guest paths before stopped-disk inspection ## Required Repo Updates diff --git a/examples/python_workspace.py b/examples/python_workspace.py index eaf0406..335c125 100644 --- a/examples/python_workspace.py +++ b/examples/python_workspace.py @@ -12,6 +12,7 @@ def main() -> None: tempfile.TemporaryDirectory(prefix="pyro-workspace-seed-") as seed_dir, tempfile.TemporaryDirectory(prefix="pyro-workspace-sync-") as sync_dir, tempfile.TemporaryDirectory(prefix="pyro-workspace-export-") as export_dir, + tempfile.TemporaryDirectory(prefix="pyro-workspace-disk-") as disk_dir, tempfile.TemporaryDirectory(prefix="pyro-workspace-secret-") as secret_dir, ): Path(seed_dir, "note.txt").write_text("hello from seed\n", encoding="utf-8") @@ -71,6 +72,17 @@ def main() -> None: service_logs = pyro.logs_service(workspace_id, "web", tail_lines=20) print(f"service_stdout_len={len(service_logs['stdout'])}") pyro.stop_service(workspace_id, "web") + stopped = pyro.stop_workspace(workspace_id) + print(f"stopped_state={stopped['state']}") + disk_listing = pyro.list_workspace_disk(workspace_id, path="/workspace", recursive=True) + print(f"disk_entries={len(disk_listing['entries'])}") + disk_read = pyro.read_workspace_disk(workspace_id, "note.txt") + print(disk_read["content"], end="") + disk_image = Path(disk_dir, "workspace.ext4") + pyro.export_workspace_disk(workspace_id, output_path=disk_image) + print(f"disk_bytes={disk_image.stat().st_size}") + started = pyro.start_workspace(workspace_id) + print(f"started_state={started['state']}") reset = pyro.reset_workspace(workspace_id, snapshot="checkpoint") print(f"reset_count={reset['reset_count']}") print(f"secret_count={len(reset['secrets'])}") diff --git a/pyproject.toml b/pyproject.toml index 9987a30..b4a1ffe 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "pyro-mcp" -version = "3.0.0" +version = "3.1.0" description = "Stable Firecracker workspaces, one-shot sandboxes, and MCP tools for coding agents." readme = "README.md" license = { file = "LICENSE" } diff --git a/src/pyro_mcp/api.py b/src/pyro_mcp/api.py index a097903..78042b7 100644 --- a/src/pyro_mcp/api.py +++ b/src/pyro_mcp/api.py @@ -119,6 +119,12 @@ class Pyro: def status_workspace(self, workspace_id: str) -> dict[str, Any]: return self._manager.status_workspace(workspace_id) + def stop_workspace(self, workspace_id: str) -> dict[str, Any]: + return self._manager.stop_workspace(workspace_id) + + def start_workspace(self, workspace_id: str) -> dict[str, Any]: + return self._manager.start_workspace(workspace_id) + def push_workspace_sync( self, workspace_id: str, @@ -151,6 +157,43 @@ class Pyro: def diff_workspace(self, workspace_id: str) -> dict[str, Any]: return self._manager.diff_workspace(workspace_id) + def export_workspace_disk( + self, + workspace_id: str, + *, + output_path: str | Path, + ) -> dict[str, Any]: + return self._manager.export_workspace_disk( + workspace_id, + output_path=output_path, + ) + + def list_workspace_disk( + self, + workspace_id: str, + *, + path: str = "/workspace", + recursive: bool = False, + ) -> dict[str, Any]: + return self._manager.list_workspace_disk( + workspace_id, + path=path, + recursive=recursive, + ) + + def read_workspace_disk( + self, + workspace_id: str, + path: str, + *, + max_bytes: int = 65536, + ) -> dict[str, Any]: + return self._manager.read_workspace_disk( + workspace_id, + path=path, + max_bytes=max_bytes, + ) + def create_snapshot(self, workspace_id: str, snapshot_name: str) -> dict[str, Any]: return self._manager.create_snapshot(workspace_id, snapshot_name) @@ -457,6 +500,16 @@ class Pyro: """Inspect workspace state and latest command metadata.""" return self.status_workspace(workspace_id) + @server.tool() + async def workspace_stop(workspace_id: str) -> dict[str, Any]: + """Stop one persistent workspace without resetting `/workspace`.""" + return self.stop_workspace(workspace_id) + + @server.tool() + async def workspace_start(workspace_id: str) -> dict[str, Any]: + """Start one stopped persistent workspace without resetting `/workspace`.""" + return self.start_workspace(workspace_id) + @server.tool() async def workspace_logs(workspace_id: str) -> dict[str, Any]: """Return persisted command history for one workspace.""" @@ -476,6 +529,40 @@ class Pyro: """Compare `/workspace` to the immutable create-time baseline.""" return self.diff_workspace(workspace_id) + @server.tool() + async def workspace_disk_export( + workspace_id: str, + output_path: str, + ) -> dict[str, Any]: + """Export the raw stopped workspace rootfs image to one host path.""" + return self.export_workspace_disk(workspace_id, output_path=output_path) + + @server.tool() + async def workspace_disk_list( + workspace_id: str, + path: str = "/workspace", + recursive: bool = False, + ) -> dict[str, Any]: + """Inspect one stopped workspace rootfs path without booting the guest.""" + return self.list_workspace_disk( + workspace_id, + path=path, + recursive=recursive, + ) + + @server.tool() + async def workspace_disk_read( + workspace_id: str, + path: str, + max_bytes: int = 65536, + ) -> dict[str, Any]: + """Read one regular file from a stopped workspace rootfs without booting the guest.""" + return self.read_workspace_disk( + workspace_id, + path, + max_bytes=max_bytes, + ) + @server.tool() async def snapshot_create(workspace_id: str, snapshot_name: str) -> dict[str, Any]: """Create one named workspace snapshot from the current `/workspace` tree.""" diff --git a/src/pyro_mcp/cli.py b/src/pyro_mcp/cli.py index 6984aec..e866053 100644 --- a/src/pyro_mcp/cli.py +++ b/src/pyro_mcp/cli.py @@ -21,6 +21,7 @@ from pyro_mcp.vm_manager import ( DEFAULT_SERVICE_READY_INTERVAL_MS, DEFAULT_SERVICE_READY_TIMEOUT_SECONDS, DEFAULT_VCPU_COUNT, + DEFAULT_WORKSPACE_DISK_READ_MAX_BYTES, WORKSPACE_GUEST_PATH, WORKSPACE_SHELL_SIGNAL_NAMES, ) @@ -253,6 +254,52 @@ def _print_workspace_export_human(payload: dict[str, Any]) -> None: ) +def _print_workspace_disk_export_human(payload: dict[str, Any]) -> None: + print( + "[workspace-disk-export] " + f"workspace_id={str(payload.get('workspace_id', 'unknown'))} " + f"output_path={str(payload.get('output_path', 'unknown'))} " + f"disk_format={str(payload.get('disk_format', 'unknown'))} " + f"bytes_written={int(payload.get('bytes_written', 0))}" + ) + + +def _print_workspace_disk_list_human(payload: dict[str, Any]) -> None: + print( + f"Workspace disk path: {str(payload.get('path', WORKSPACE_GUEST_PATH))} " + f"(recursive={'yes' if bool(payload.get('recursive')) else 'no'})" + ) + entries = payload.get("entries") + if not isinstance(entries, list) or not entries: + print("No workspace disk entries found.") + return + for entry in entries: + if not isinstance(entry, dict): + continue + line = ( + f"{str(entry.get('path', 'unknown'))} " + f"[{str(entry.get('artifact_type', 'unknown'))}] " + f"size={int(entry.get('size_bytes', 0))}" + ) + link_target = entry.get("link_target") + if isinstance(link_target, str) and link_target != "": + line += f" -> {link_target}" + print(line) + + +def _print_workspace_disk_read_human(payload: dict[str, Any]) -> None: + _write_stream(str(payload.get("content", "")), stream=sys.stdout) + print( + "[workspace-disk-read] " + f"workspace_id={str(payload.get('workspace_id', 'unknown'))} " + f"path={str(payload.get('path', 'unknown'))} " + f"size_bytes={int(payload.get('size_bytes', 0))} " + f"truncated={'yes' if bool(payload.get('truncated', False)) else 'no'}", + file=sys.stderr, + flush=True, + ) + + def _print_workspace_diff_human(payload: dict[str, Any]) -> None: if not bool(payload.get("changed")): print("No workspace changes.") @@ -687,6 +734,10 @@ def _build_parser() -> argparse.ArgumentParser: pyro workspace create debian:12 --seed-path ./repo pyro workspace sync push WORKSPACE_ID ./repo --dest src pyro workspace exec WORKSPACE_ID -- sh -lc 'printf "hello\\n" > note.txt' + pyro workspace stop WORKSPACE_ID + pyro workspace disk list WORKSPACE_ID + pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4 + pyro workspace start WORKSPACE_ID pyro workspace snapshot create WORKSPACE_ID checkpoint pyro workspace reset WORKSPACE_ID --snapshot checkpoint pyro workspace diff WORKSPACE_ID @@ -1039,6 +1090,141 @@ def _build_parser() -> argparse.ArgumentParser: action="store_true", help="Print structured JSON instead of human-readable output.", ) + workspace_stop_parser = workspace_subparsers.add_parser( + "stop", + help="Stop one workspace without resetting it.", + description=( + "Stop the backing sandbox, close shells, stop services, and preserve the " + "workspace filesystem, history, and snapshots." + ), + epilog="Example:\n pyro workspace stop WORKSPACE_ID", + formatter_class=_HelpFormatter, + ) + workspace_stop_parser.add_argument( + "workspace_id", + metavar="WORKSPACE_ID", + help="Persistent workspace identifier.", + ) + workspace_stop_parser.add_argument( + "--json", + action="store_true", + help="Print structured JSON instead of human-readable output.", + ) + workspace_start_parser = workspace_subparsers.add_parser( + "start", + help="Start one stopped workspace without resetting it.", + description=( + "Start a previously stopped workspace from its preserved rootfs and " + "workspace state." + ), + epilog="Example:\n pyro workspace start WORKSPACE_ID", + formatter_class=_HelpFormatter, + ) + workspace_start_parser.add_argument( + "workspace_id", + metavar="WORKSPACE_ID", + help="Persistent workspace identifier.", + ) + workspace_start_parser.add_argument( + "--json", + action="store_true", + help="Print structured JSON instead of human-readable output.", + ) + workspace_disk_parser = workspace_subparsers.add_parser( + "disk", + help="Inspect or export a stopped workspace disk.", + description=( + "Use secondary stopped-workspace disk tools for raw ext4 export and offline " + "inspection without booting the guest." + ), + epilog=dedent( + """ + Examples: + pyro workspace stop WORKSPACE_ID + pyro workspace disk list WORKSPACE_ID + pyro workspace disk read WORKSPACE_ID note.txt + pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4 + + Disk tools are secondary to `workspace export` and require a stopped, guest-backed + workspace. + """ + ), + formatter_class=_HelpFormatter, + ) + workspace_disk_subparsers = workspace_disk_parser.add_subparsers( + dest="workspace_disk_command", + required=True, + metavar="DISK", + ) + workspace_disk_export_parser = workspace_disk_subparsers.add_parser( + "export", + help="Export the raw stopped workspace rootfs image.", + description="Copy the raw stopped workspace rootfs ext4 image to an explicit host path.", + epilog="Example:\n pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4", + formatter_class=_HelpFormatter, + ) + workspace_disk_export_parser.add_argument("workspace_id", metavar="WORKSPACE_ID") + workspace_disk_export_parser.add_argument( + "--output", + required=True, + help="Exact host path to create for the exported raw ext4 image.", + ) + workspace_disk_export_parser.add_argument( + "--json", + action="store_true", + help="Print structured JSON instead of human-readable output.", + ) + workspace_disk_list_parser = workspace_disk_subparsers.add_parser( + "list", + help="List files from a stopped workspace rootfs path.", + description=( + "Inspect one stopped workspace rootfs path without booting the guest. Relative " + "paths resolve inside `/workspace`; absolute paths inspect any guest path." + ), + epilog="Example:\n pyro workspace disk list WORKSPACE_ID src --recursive", + formatter_class=_HelpFormatter, + ) + workspace_disk_list_parser.add_argument("workspace_id", metavar="WORKSPACE_ID") + workspace_disk_list_parser.add_argument( + "path", + nargs="?", + default=WORKSPACE_GUEST_PATH, + metavar="PATH", + help="Guest path to inspect. Defaults to `/workspace`.", + ) + workspace_disk_list_parser.add_argument( + "--recursive", + action="store_true", + help="Recurse into nested directories.", + ) + workspace_disk_list_parser.add_argument( + "--json", + action="store_true", + help="Print structured JSON instead of human-readable output.", + ) + workspace_disk_read_parser = workspace_disk_subparsers.add_parser( + "read", + help="Read one regular file from a stopped workspace rootfs.", + description=( + "Read one regular file from a stopped workspace rootfs without booting the guest. " + "Relative paths resolve inside `/workspace`; absolute paths inspect any guest path." + ), + epilog="Example:\n pyro workspace disk read WORKSPACE_ID note.txt --max-bytes 4096", + formatter_class=_HelpFormatter, + ) + workspace_disk_read_parser.add_argument("workspace_id", metavar="WORKSPACE_ID") + workspace_disk_read_parser.add_argument("path", metavar="PATH") + workspace_disk_read_parser.add_argument( + "--max-bytes", + type=int, + default=DEFAULT_WORKSPACE_DISK_READ_MAX_BYTES, + help="Maximum number of decoded UTF-8 bytes to return.", + ) + workspace_disk_read_parser.add_argument( + "--json", + action="store_true", + help="Print structured JSON instead of human-readable output.", + ) workspace_shell_parser = workspace_subparsers.add_parser( "shell", help="Open and manage persistent interactive shells.", @@ -1885,6 +2071,88 @@ def main() -> None: else: _print_workspace_reset_human(payload) return + if args.workspace_command == "stop": + try: + payload = pyro.stop_workspace(args.workspace_id) + except Exception as exc: # noqa: BLE001 + if bool(args.json): + _print_json({"ok": False, "error": str(exc)}) + else: + print(f"[error] {exc}", file=sys.stderr, flush=True) + raise SystemExit(1) from exc + if bool(args.json): + _print_json(payload) + else: + _print_workspace_summary_human(payload, action="Stopped workspace") + return + if args.workspace_command == "start": + try: + payload = pyro.start_workspace(args.workspace_id) + except Exception as exc: # noqa: BLE001 + if bool(args.json): + _print_json({"ok": False, "error": str(exc)}) + else: + print(f"[error] {exc}", file=sys.stderr, flush=True) + raise SystemExit(1) from exc + if bool(args.json): + _print_json(payload) + else: + _print_workspace_summary_human(payload, action="Started workspace") + return + if args.workspace_command == "disk": + if args.workspace_disk_command == "export": + try: + payload = pyro.export_workspace_disk( + args.workspace_id, + output_path=args.output, + ) + except Exception as exc: # noqa: BLE001 + if bool(args.json): + _print_json({"ok": False, "error": str(exc)}) + else: + print(f"[error] {exc}", file=sys.stderr, flush=True) + raise SystemExit(1) from exc + if bool(args.json): + _print_json(payload) + else: + _print_workspace_disk_export_human(payload) + return + if args.workspace_disk_command == "list": + try: + payload = pyro.list_workspace_disk( + args.workspace_id, + path=args.path, + recursive=bool(args.recursive), + ) + except Exception as exc: # noqa: BLE001 + if bool(args.json): + _print_json({"ok": False, "error": str(exc)}) + else: + print(f"[error] {exc}", file=sys.stderr, flush=True) + raise SystemExit(1) from exc + if bool(args.json): + _print_json(payload) + else: + _print_workspace_disk_list_human(payload) + return + if args.workspace_disk_command == "read": + try: + payload = pyro.read_workspace_disk( + args.workspace_id, + args.path, + max_bytes=args.max_bytes, + ) + except Exception as exc: # noqa: BLE001 + if bool(args.json): + _print_json({"ok": False, "error": str(exc)}) + else: + print(f"[error] {exc}", file=sys.stderr, flush=True) + raise SystemExit(1) from exc + if bool(args.json): + _print_json(payload) + else: + _print_workspace_disk_read_human(payload) + return if args.workspace_command == "shell": if args.workspace_shell_command == "open": secret_env = _parse_workspace_secret_env_options(getattr(args, "secret_env", [])) diff --git a/src/pyro_mcp/contract.py b/src/pyro_mcp/contract.py index d952a8b..aeef937 100644 --- a/src/pyro_mcp/contract.py +++ b/src/pyro_mcp/contract.py @@ -8,6 +8,7 @@ PUBLIC_CLI_ENV_SUBCOMMANDS = ("inspect", "list", "pull", "prune") PUBLIC_CLI_WORKSPACE_SUBCOMMANDS = ( "create", "delete", + "disk", "diff", "exec", "export", @@ -16,9 +17,12 @@ PUBLIC_CLI_WORKSPACE_SUBCOMMANDS = ( "service", "shell", "snapshot", + "start", "status", + "stop", "sync", ) +PUBLIC_CLI_WORKSPACE_DISK_SUBCOMMANDS = ("export", "list", "read") PUBLIC_CLI_WORKSPACE_SERVICE_SUBCOMMANDS = ("list", "logs", "start", "status", "stop") PUBLIC_CLI_WORKSPACE_SHELL_SUBCOMMANDS = ("close", "open", "read", "signal", "write") PUBLIC_CLI_WORKSPACE_SNAPSHOT_SUBCOMMANDS = ("create", "delete", "list") @@ -34,6 +38,9 @@ PUBLIC_CLI_WORKSPACE_CREATE_FLAGS = ( "--secret-file", "--json", ) +PUBLIC_CLI_WORKSPACE_DISK_EXPORT_FLAGS = ("--output", "--json") +PUBLIC_CLI_WORKSPACE_DISK_LIST_FLAGS = ("--recursive", "--json") +PUBLIC_CLI_WORKSPACE_DISK_READ_FLAGS = ("--max-bytes", "--json") PUBLIC_CLI_WORKSPACE_EXEC_FLAGS = ("--timeout-seconds", "--secret-env", "--json") PUBLIC_CLI_WORKSPACE_DIFF_FLAGS = ("--json",) PUBLIC_CLI_WORKSPACE_EXPORT_FLAGS = ("--output", "--json") @@ -68,6 +75,9 @@ PUBLIC_CLI_WORKSPACE_SHELL_CLOSE_FLAGS = ("--json",) PUBLIC_CLI_WORKSPACE_SNAPSHOT_CREATE_FLAGS = ("--json",) PUBLIC_CLI_WORKSPACE_SNAPSHOT_DELETE_FLAGS = ("--json",) PUBLIC_CLI_WORKSPACE_SNAPSHOT_LIST_FLAGS = ("--json",) +PUBLIC_CLI_WORKSPACE_START_FLAGS = ("--json",) +PUBLIC_CLI_WORKSPACE_STATUS_FLAGS = ("--json",) +PUBLIC_CLI_WORKSPACE_STOP_FLAGS = ("--json",) PUBLIC_CLI_WORKSPACE_SYNC_PUSH_FLAGS = ("--dest", "--json") PUBLIC_CLI_RUN_FLAGS = ( "--vcpu-count", @@ -92,10 +102,12 @@ PUBLIC_SDK_METHODS = ( "exec_vm", "exec_workspace", "export_workspace", + "export_workspace_disk", "inspect_environment", "list_environments", "list_services", "list_snapshots", + "list_workspace_disk", "logs_service", "logs_workspace", "network_info_vm", @@ -104,17 +116,20 @@ PUBLIC_SDK_METHODS = ( "pull_environment", "push_workspace_sync", "read_shell", + "read_workspace_disk", "reap_expired", "reset_workspace", "run_in_vm", "signal_shell", "start_service", "start_vm", + "start_workspace", "status_service", "status_vm", "status_workspace", "stop_service", "stop_vm", + "stop_workspace", "write_shell", ) @@ -144,11 +159,16 @@ PUBLIC_MCP_TOOLS = ( "vm_stop", "workspace_create", "workspace_delete", + "workspace_disk_export", + "workspace_disk_list", + "workspace_disk_read", "workspace_diff", "workspace_exec", "workspace_export", "workspace_logs", "workspace_reset", + "workspace_start", "workspace_status", + "workspace_stop", "workspace_sync_push", ) diff --git a/src/pyro_mcp/vm_environments.py b/src/pyro_mcp/vm_environments.py index 1cedeeb..c34537e 100644 --- a/src/pyro_mcp/vm_environments.py +++ b/src/pyro_mcp/vm_environments.py @@ -19,7 +19,7 @@ from typing import Any from pyro_mcp.runtime import DEFAULT_PLATFORM, RuntimePaths DEFAULT_ENVIRONMENT_VERSION = "1.0.0" -DEFAULT_CATALOG_VERSION = "3.0.0" +DEFAULT_CATALOG_VERSION = "3.1.0" OCI_MANIFEST_ACCEPT = ", ".join( ( "application/vnd.oci.image.index.v1+json", diff --git a/src/pyro_mcp/vm_manager.py b/src/pyro_mcp/vm_manager.py index 3dad54d..20e7704 100644 --- a/src/pyro_mcp/vm_manager.py +++ b/src/pyro_mcp/vm_manager.py @@ -34,6 +34,12 @@ from pyro_mcp.vm_environments import EnvironmentStore, default_cache_dir, get_en from pyro_mcp.vm_firecracker import build_launch_plan from pyro_mcp.vm_guest import VsockExecClient from pyro_mcp.vm_network import NetworkConfig, TapNetworkManager +from pyro_mcp.workspace_disk import ( + export_workspace_disk_image, + list_workspace_disk, + read_workspace_disk_file, + scrub_workspace_runtime_paths, +) from pyro_mcp.workspace_ports import DEFAULT_PUBLISHED_PORT_HOST from pyro_mcp.workspace_shells import ( create_local_shell, @@ -72,6 +78,7 @@ WORKSPACE_SECRET_MAX_BYTES = 64 * 1024 DEFAULT_SHELL_COLS = 120 DEFAULT_SHELL_ROWS = 30 DEFAULT_SHELL_MAX_CHARS = 65536 +DEFAULT_WORKSPACE_DISK_READ_MAX_BYTES = 65536 DEFAULT_SERVICE_READY_TIMEOUT_SECONDS = 30 DEFAULT_SERVICE_READY_INTERVAL_MS = 500 DEFAULT_SERVICE_LOG_TAIL_LINES = 200 @@ -789,6 +796,28 @@ def _workspace_host_destination(workspace_dir: Path, destination: str) -> Path: return workspace_dir.joinpath(*suffix.parts) +def _normalize_workspace_disk_path(path: str) -> str: + candidate = path.strip() + if candidate == "": + raise ValueError("workspace disk path must not be empty") + if candidate.startswith("/"): + raw_path = PurePosixPath(candidate) + normalized_parts: list[str] = [] + for part in raw_path.parts: + if part in {"", "/", "."}: + continue + if part == "..": + if normalized_parts: + normalized_parts.pop() + continue + normalized_parts.append(part) + if not normalized_parts: + return "/" + return str(PurePosixPath("/") / PurePosixPath(*normalized_parts)) + normalized, _ = _normalize_workspace_destination(candidate) + return normalized + + def _normalize_archive_member_name(name: str) -> PurePosixPath: candidate = name.strip() if candidate == "": @@ -2480,6 +2509,11 @@ class FirecrackerBackend(VmBackend): # pragma: no cover def start(self, instance: VmInstance) -> None: launch_plan = build_launch_plan(instance) + for stale_socket_path in ( + launch_plan.api_socket_path, + instance.workdir / "vsock.sock", + ): + stale_socket_path.unlink(missing_ok=True) instance.metadata["firecracker_config_path"] = str(launch_plan.config_path) instance.metadata["guest_network_path"] = str(launch_plan.guest_network_path) instance.metadata["guest_exec_path"] = str(launch_plan.guest_exec_path) @@ -4309,6 +4343,159 @@ class VmManager: "entries": redacted_entries, } + def stop_workspace(self, workspace_id: str) -> dict[str, Any]: + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + self._ensure_workspace_not_expired_locked(workspace, time.time()) + self._refresh_workspace_liveness_locked(workspace) + instance = workspace.to_instance( + workdir=self._workspace_runtime_dir(workspace.workspace_id) + ) + try: + self._stop_workspace_services_locked(workspace, instance) + self._close_workspace_shells_locked(workspace, instance) + self._flush_workspace_filesystem_locked(workspace, instance) + if workspace.state == "started": + self._backend.stop(instance) + workspace.state = "stopped" + workspace.firecracker_pid = None + workspace.last_error = None + workspace.metadata = dict(instance.metadata) + self._scrub_workspace_runtime_state_locked(workspace) + except Exception as exc: + workspace.state = "stopped" + workspace.firecracker_pid = None + workspace.last_error = str(exc) + workspace.metadata = dict(instance.metadata) + self._save_workspace_locked(workspace) + raise + self._save_workspace_locked(workspace) + return self._serialize_workspace(workspace) + + def start_workspace(self, workspace_id: str) -> dict[str, Any]: + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + self._ensure_workspace_not_expired_locked(workspace, time.time()) + self._refresh_workspace_liveness_locked(workspace) + if workspace.state == "started": + self._refresh_workspace_service_counts_locked(workspace) + self._save_workspace_locked(workspace) + return self._serialize_workspace(workspace) + instance = workspace.to_instance( + workdir=self._workspace_runtime_dir(workspace.workspace_id) + ) + self._stop_workspace_services_locked(workspace, instance) + self._close_workspace_shells_locked(workspace, instance) + try: + self._require_workspace_network_policy_support( + network_policy=workspace.network_policy + ) + if self._runtime_capabilities.supports_guest_exec: + self._ensure_workspace_guest_bootstrap_support(instance) + with self._lock: + self._start_instance_locked(instance) + workspace = self._load_workspace_locked(workspace_id) + if workspace.secrets: + self._install_workspace_secrets_locked(workspace, instance) + workspace.state = instance.state + workspace.firecracker_pid = instance.firecracker_pid + workspace.last_error = None + workspace.metadata = dict(instance.metadata) + self._save_workspace_locked(workspace) + return self._serialize_workspace(workspace) + except Exception as exc: + try: + if instance.state == "started": + self._backend.stop(instance) + except Exception: + pass + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + workspace.state = "stopped" + workspace.firecracker_pid = None + workspace.last_error = str(exc) + workspace.metadata = dict(instance.metadata) + self._save_workspace_locked(workspace) + raise + + def export_workspace_disk( + self, + workspace_id: str, + *, + output_path: str | Path, + ) -> dict[str, Any]: + raw_output_path = str(output_path).strip() + if raw_output_path == "": + raise ValueError("output_path must not be empty") + resolved_output_path = Path(output_path).expanduser().resolve() + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + rootfs_path = self._workspace_stopped_disk_rootfs_locked( + workspace, + operation_name="workspace_disk_export", + ) + self._scrub_workspace_runtime_state_locked(workspace, rootfs_path=rootfs_path) + self._save_workspace_locked(workspace) + exported = export_workspace_disk_image(rootfs_path, output_path=resolved_output_path) + return { + "workspace_id": workspace_id, + "output_path": str(Path(str(exported["output_path"]))), + "disk_format": str(exported["disk_format"]), + "bytes_written": int(exported["bytes_written"]), + } + + def list_workspace_disk( + self, + workspace_id: str, + *, + path: str = WORKSPACE_GUEST_PATH, + recursive: bool = False, + ) -> dict[str, Any]: + normalized_path = _normalize_workspace_disk_path(path) + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + rootfs_path = self._workspace_stopped_disk_rootfs_locked( + workspace, + operation_name="workspace_disk_list", + ) + self._scrub_workspace_runtime_state_locked(workspace, rootfs_path=rootfs_path) + self._save_workspace_locked(workspace) + entries = list_workspace_disk( + rootfs_path, + guest_path=normalized_path, + recursive=recursive, + ) + return { + "workspace_id": workspace_id, + "path": normalized_path, + "recursive": recursive, + "entries": entries, + } + + def read_workspace_disk( + self, + workspace_id: str, + *, + path: str, + max_bytes: int = DEFAULT_WORKSPACE_DISK_READ_MAX_BYTES, + ) -> dict[str, Any]: + normalized_path = _normalize_workspace_disk_path(path) + with self._lock: + workspace = self._load_workspace_locked(workspace_id) + rootfs_path = self._workspace_stopped_disk_rootfs_locked( + workspace, + operation_name="workspace_disk_read", + ) + self._scrub_workspace_runtime_state_locked(workspace, rootfs_path=rootfs_path) + self._save_workspace_locked(workspace) + payload = read_workspace_disk_file( + rootfs_path, + guest_path=normalized_path, + max_bytes=max_bytes, + ) + payload["workspace_id"] = workspace_id + return payload + def delete_workspace( self, workspace_id: str, @@ -4748,6 +4935,67 @@ class VmManager: def _workspace_service_record_path(self, workspace_id: str, service_name: str) -> Path: return self._workspace_services_dir(workspace_id) / f"{service_name}.json" + def _workspace_rootfs_image_path_locked( + self, + workspace: WorkspaceRecord, + ) -> Path: + raw_rootfs_image = workspace.metadata.get("rootfs_image") + if raw_rootfs_image is None or raw_rootfs_image == "": + raise RuntimeError( + f"workspace {workspace.workspace_id!r} does not have a persisted rootfs image" + ) + rootfs_path = Path(raw_rootfs_image) + if not rootfs_path.exists(): + raise RuntimeError( + f"workspace {workspace.workspace_id!r} rootfs image is unavailable at " + f"{rootfs_path}" + ) + return rootfs_path + + def _workspace_stopped_disk_rootfs_locked( + self, + workspace: WorkspaceRecord, + *, + operation_name: str, + ) -> Path: + self._ensure_workspace_not_expired_locked(workspace, time.time()) + self._refresh_workspace_liveness_locked(workspace) + if workspace.state != "stopped": + raise RuntimeError( + f"workspace {workspace.workspace_id!r} must be stopped before {operation_name}" + ) + if workspace.metadata.get("execution_mode") == "host_compat": + raise RuntimeError( + f"{operation_name} is unavailable for host_compat workspaces" + ) + return self._workspace_rootfs_image_path_locked(workspace) + + def _scrub_workspace_runtime_state_locked( + self, + workspace: WorkspaceRecord, + *, + rootfs_path: Path | None = None, + ) -> None: + execution_mode = workspace.metadata.get("execution_mode") + if execution_mode == "host_compat": + return + scrub_workspace_runtime_paths( + rootfs_path or self._workspace_rootfs_image_path_locked(workspace) + ) + + def _flush_workspace_filesystem_locked( + self, + workspace: WorkspaceRecord, + instance: VmInstance, + ) -> None: + if workspace.state != "started": + return + if self._backend_name == "mock": + return + if not self._runtime_capabilities.supports_guest_exec: + return + self._backend.exec(instance, "sync", 10) + def _count_workspaces_locked(self) -> int: return sum(1 for _ in self._workspaces_dir.glob("*/workspace.json")) diff --git a/src/pyro_mcp/workspace_disk.py b/src/pyro_mcp/workspace_disk.py new file mode 100644 index 0000000..756cfac --- /dev/null +++ b/src/pyro_mcp/workspace_disk.py @@ -0,0 +1,264 @@ +"""Stopped-workspace disk export and offline inspection helpers.""" + +from __future__ import annotations + +import re +import shutil +import subprocess +import tempfile +from dataclasses import dataclass +from pathlib import Path, PurePosixPath +from typing import Literal + +WorkspaceDiskArtifactType = Literal["file", "directory", "symlink"] + +WORKSPACE_DISK_RUNTIME_ONLY_PATHS = ( + "/run/pyro-secrets", + "/run/pyro-shells", + "/run/pyro-services", +) + +_DEBUGFS_LS_RE = re.compile( + r"^/(?P\d+)/(?P\d+)/(?P\d+)/(?P\d+)/(?P.*)/(?P\d*)/$" +) +_DEBUGFS_SIZE_RE = re.compile(r"Size:\s+(?P\d+)") +_DEBUGFS_TYPE_RE = re.compile(r"Type:\s+(?P\w+)") +_DEBUGFS_LINK_RE = re.compile(r'Fast link dest:\s+"(?P.*)"') + + +@dataclass(frozen=True) +class WorkspaceDiskEntry: + """One inspectable path from a stopped workspace rootfs image.""" + + path: str + artifact_type: WorkspaceDiskArtifactType + size_bytes: int + link_target: str | None = None + + def to_payload(self) -> dict[str, str | int | None]: + return { + "path": self.path, + "artifact_type": self.artifact_type, + "size_bytes": self.size_bytes, + "link_target": self.link_target, + } + + +@dataclass(frozen=True) +class _DebugfsStat: + path: str + artifact_type: WorkspaceDiskArtifactType + size_bytes: int + link_target: str | None = None + + +@dataclass(frozen=True) +class _DebugfsDirEntry: + name: str + path: str + artifact_type: WorkspaceDiskArtifactType | None + size_bytes: int + + +def export_workspace_disk_image(rootfs_image: Path, *, output_path: Path) -> dict[str, str | int]: + """Copy one stopped workspace rootfs image to the requested host path.""" + output_path.parent.mkdir(parents=True, exist_ok=True) + if output_path.exists() or output_path.is_symlink(): + raise RuntimeError(f"output_path already exists: {output_path}") + shutil.copy2(rootfs_image, output_path) + return { + "output_path": str(output_path), + "disk_format": "ext4", + "bytes_written": output_path.stat().st_size, + } + + +def list_workspace_disk( + rootfs_image: Path, + *, + guest_path: str, + recursive: bool, +) -> list[dict[str, str | int | None]]: + """Return inspectable entries from one stopped workspace rootfs path.""" + target = _debugfs_stat(rootfs_image, guest_path) + if target is None: + raise RuntimeError(f"workspace disk path does not exist: {guest_path}") + if target.artifact_type != "directory": + return [WorkspaceDiskEntry(**target.__dict__).to_payload()] + entries: list[WorkspaceDiskEntry] = [] + + def walk(current_path: str) -> None: + children = _debugfs_ls_entries(rootfs_image, current_path) + for child in children: + if child.artifact_type is None: + continue + link_target = None + if child.artifact_type == "symlink": + child_stat = _debugfs_stat(rootfs_image, child.path) + link_target = None if child_stat is None else child_stat.link_target + entries.append( + WorkspaceDiskEntry( + path=child.path, + artifact_type=child.artifact_type, + size_bytes=child.size_bytes, + link_target=link_target, + ) + ) + if recursive and child.artifact_type == "directory": + walk(child.path) + + walk(guest_path) + entries.sort(key=lambda item: item.path) + return [entry.to_payload() for entry in entries] + + +def read_workspace_disk_file( + rootfs_image: Path, + *, + guest_path: str, + max_bytes: int, +) -> dict[str, str | int | bool]: + """Read one regular file from a stopped workspace rootfs image.""" + target = _debugfs_stat(rootfs_image, guest_path) + if target is None: + raise RuntimeError(f"workspace disk path does not exist: {guest_path}") + if target.artifact_type != "file": + raise RuntimeError("workspace disk read only supports regular files") + if max_bytes <= 0: + raise ValueError("max_bytes must be positive") + with tempfile.TemporaryDirectory(prefix="pyro-workspace-disk-read-") as temp_dir: + dumped_path = Path(temp_dir) / "workspace-disk-read.bin" + _run_debugfs(rootfs_image, f"dump {guest_path} {dumped_path}") + if not dumped_path.exists(): + raise RuntimeError(f"failed to dump workspace disk file: {guest_path}") + raw_bytes = dumped_path.read_bytes() + return { + "path": guest_path, + "size_bytes": len(raw_bytes), + "max_bytes": max_bytes, + "content": raw_bytes[:max_bytes].decode("utf-8", errors="replace"), + "truncated": len(raw_bytes) > max_bytes, + } + + +def scrub_workspace_runtime_paths(rootfs_image: Path) -> None: + """Remove runtime-only guest paths from a stopped workspace rootfs image.""" + for guest_path in WORKSPACE_DISK_RUNTIME_ONLY_PATHS: + _debugfs_remove_tree(rootfs_image, guest_path) + + +def _run_debugfs(rootfs_image: Path, command: str, *, writable: bool = False) -> str: + debugfs_path = shutil.which("debugfs") + if debugfs_path is None: + raise RuntimeError("debugfs is required for workspace disk operations") + debugfs_command = [debugfs_path] + if writable: + debugfs_command.append("-w") + proc = subprocess.run( # noqa: S603 + [*debugfs_command, "-R", command, str(rootfs_image)], + text=True, + capture_output=True, + check=False, + ) + combined = proc.stdout + if proc.stderr != "": + combined = combined + ("\n" if combined != "" else "") + proc.stderr + output = _strip_debugfs_banner(combined) + if proc.returncode != 0: + message = output.strip() + if message == "": + message = f"debugfs command failed: {command}" + raise RuntimeError(message) + return output.strip() + + +def _strip_debugfs_banner(output: str) -> str: + lines = output.splitlines() + while lines and lines[0].startswith("debugfs "): + lines.pop(0) + return "\n".join(lines) + + +def _debugfs_missing(output: str) -> bool: + return "File not found by ext2_lookup" in output or "File not found by ext2fs_lookup" in output + + +def _artifact_type_from_mode(mode: str) -> WorkspaceDiskArtifactType | None: + if mode.startswith("04"): + return "directory" + if mode.startswith("10"): + return "file" + if mode.startswith("12"): + return "symlink" + return None + + +def _debugfs_stat(rootfs_image: Path, guest_path: str) -> _DebugfsStat | None: + output = _run_debugfs(rootfs_image, f"stat {guest_path}") + if _debugfs_missing(output): + return None + type_match = _DEBUGFS_TYPE_RE.search(output) + size_match = _DEBUGFS_SIZE_RE.search(output) + if type_match is None or size_match is None: + raise RuntimeError(f"failed to inspect workspace disk path: {guest_path}") + raw_type = type_match.group("type") + artifact_type: WorkspaceDiskArtifactType + if raw_type == "directory": + artifact_type = "directory" + elif raw_type == "regular": + artifact_type = "file" + elif raw_type == "symlink": + artifact_type = "symlink" + else: + raise RuntimeError(f"unsupported workspace disk path type: {guest_path}") + link_target = None + if artifact_type == "symlink": + link_match = _DEBUGFS_LINK_RE.search(output) + if link_match is not None: + link_target = link_match.group("target") + return _DebugfsStat( + path=guest_path, + artifact_type=artifact_type, + size_bytes=int(size_match.group("size")), + link_target=link_target, + ) + + +def _debugfs_ls_entries(rootfs_image: Path, guest_path: str) -> list[_DebugfsDirEntry]: + output = _run_debugfs(rootfs_image, f"ls -p {guest_path}") + if _debugfs_missing(output): + raise RuntimeError(f"workspace disk path does not exist: {guest_path}") + entries: list[_DebugfsDirEntry] = [] + base = PurePosixPath(guest_path) + for raw_line in output.splitlines(): + line = raw_line.strip() + if line == "": + continue + match = _DEBUGFS_LS_RE.match(line) + if match is None: + continue + name = match.group("name") + if name in {".", ".."}: + continue + child_path = str(base / name) if str(base) != "/" else f"/{name}" + entries.append( + _DebugfsDirEntry( + name=name, + path=child_path, + artifact_type=_artifact_type_from_mode(match.group("mode")), + size_bytes=int(match.group("size") or "0"), + ) + ) + return entries + + +def _debugfs_remove_tree(rootfs_image: Path, guest_path: str) -> None: + stat_result = _debugfs_stat(rootfs_image, guest_path) + if stat_result is None: + return + if stat_result.artifact_type == "directory": + for child in _debugfs_ls_entries(rootfs_image, guest_path): + _debugfs_remove_tree(rootfs_image, child.path) + _run_debugfs(rootfs_image, f"rmdir {guest_path}", writable=True) + return + _run_debugfs(rootfs_image, f"rm {guest_path}", writable=True) diff --git a/tests/test_api.py b/tests/test_api.py index 61b7e86..3ae927a 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -50,9 +50,14 @@ def test_pyro_create_server_registers_vm_run(tmp_path: Path) -> None: assert "vm_run" in tool_names assert "vm_create" in tool_names assert "workspace_create" in tool_names + assert "workspace_start" in tool_names + assert "workspace_stop" in tool_names assert "workspace_diff" in tool_names assert "workspace_sync_push" in tool_names assert "workspace_export" in tool_names + assert "workspace_disk_export" in tool_names + assert "workspace_disk_list" in tool_names + assert "workspace_disk_read" in tool_names assert "snapshot_create" in tool_names assert "snapshot_list" in tool_names assert "snapshot_delete" in tool_names @@ -289,3 +294,603 @@ def test_pyro_workspace_methods_delegate_to_manager(tmp_path: Path) -> None: assert status["service_count"] == 0 assert logs["count"] == 0 assert deleted["deleted"] is True + + +def test_pyro_workspace_disk_methods_delegate_to_manager() -> None: + calls: list[tuple[str, dict[str, Any]]] = [] + + class StubManager: + def stop_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("stop_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "state": "stopped"} + + def start_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("start_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "state": "started"} + + def export_workspace_disk(self, workspace_id: str, *, output_path: Path) -> dict[str, Any]: + calls.append( + ( + "export_workspace_disk", + {"workspace_id": workspace_id, "output_path": str(output_path)}, + ) + ) + return {"workspace_id": workspace_id, "output_path": str(output_path)} + + def list_workspace_disk( + self, + workspace_id: str, + *, + path: str = "/workspace", + recursive: bool = False, + ) -> dict[str, Any]: + calls.append( + ( + "list_workspace_disk", + { + "workspace_id": workspace_id, + "path": path, + "recursive": recursive, + }, + ) + ) + return {"workspace_id": workspace_id, "entries": []} + + def read_workspace_disk( + self, + workspace_id: str, + *, + path: str, + max_bytes: int = 65536, + ) -> dict[str, Any]: + calls.append( + ( + "read_workspace_disk", + { + "workspace_id": workspace_id, + "path": path, + "max_bytes": max_bytes, + }, + ) + ) + return {"workspace_id": workspace_id, "content": ""} + + pyro = Pyro(manager=cast(Any, StubManager())) + + stopped = pyro.stop_workspace("workspace-123") + started = pyro.start_workspace("workspace-123") + exported = pyro.export_workspace_disk("workspace-123", output_path=Path("/tmp/workspace.ext4")) + listed = pyro.list_workspace_disk("workspace-123", path="/workspace/src", recursive=True) + read = pyro.read_workspace_disk("workspace-123", "note.txt", max_bytes=4096) + + assert stopped["state"] == "stopped" + assert started["state"] == "started" + assert exported["output_path"] == "/tmp/workspace.ext4" + assert listed["entries"] == [] + assert read["content"] == "" + assert calls == [ + ("stop_workspace", {"workspace_id": "workspace-123"}), + ("start_workspace", {"workspace_id": "workspace-123"}), + ( + "export_workspace_disk", + { + "workspace_id": "workspace-123", + "output_path": "/tmp/workspace.ext4", + }, + ), + ( + "list_workspace_disk", + { + "workspace_id": "workspace-123", + "path": "/workspace/src", + "recursive": True, + }, + ), + ( + "read_workspace_disk", + { + "workspace_id": "workspace-123", + "path": "note.txt", + "max_bytes": 4096, + }, + ), + ] + + +def test_pyro_create_server_workspace_disk_tools_delegate() -> None: + calls: list[tuple[str, dict[str, Any]]] = [] + + class StubManager: + def stop_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("stop_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "state": "stopped"} + + def start_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("start_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "state": "started"} + + def export_workspace_disk(self, workspace_id: str, *, output_path: str) -> dict[str, Any]: + calls.append( + ( + "export_workspace_disk", + {"workspace_id": workspace_id, "output_path": output_path}, + ) + ) + return {"workspace_id": workspace_id, "output_path": output_path} + + def list_workspace_disk( + self, + workspace_id: str, + *, + path: str = "/workspace", + recursive: bool = False, + ) -> dict[str, Any]: + calls.append( + ( + "list_workspace_disk", + { + "workspace_id": workspace_id, + "path": path, + "recursive": recursive, + }, + ) + ) + return {"workspace_id": workspace_id, "entries": []} + + def read_workspace_disk( + self, + workspace_id: str, + *, + path: str, + max_bytes: int = 65536, + ) -> dict[str, Any]: + calls.append( + ( + "read_workspace_disk", + { + "workspace_id": workspace_id, + "path": path, + "max_bytes": max_bytes, + }, + ) + ) + return {"workspace_id": workspace_id, "content": ""} + + pyro = Pyro(manager=cast(Any, StubManager())) + + def _extract_structured(raw_result: object) -> dict[str, Any]: + if not isinstance(raw_result, tuple) or len(raw_result) != 2: + raise TypeError("unexpected call_tool result shape") + _, structured = raw_result + if not isinstance(structured, dict): + raise TypeError("expected structured dictionary result") + return cast(dict[str, Any], structured) + + async def _run() -> tuple[dict[str, Any], ...]: + server = pyro.create_server() + stopped = _extract_structured( + await server.call_tool("workspace_stop", {"workspace_id": "workspace-123"}) + ) + started = _extract_structured( + await server.call_tool("workspace_start", {"workspace_id": "workspace-123"}) + ) + exported = _extract_structured( + await server.call_tool( + "workspace_disk_export", + { + "workspace_id": "workspace-123", + "output_path": "/tmp/workspace.ext4", + }, + ) + ) + listed = _extract_structured( + await server.call_tool( + "workspace_disk_list", + { + "workspace_id": "workspace-123", + "path": "/workspace/src", + "recursive": True, + }, + ) + ) + read = _extract_structured( + await server.call_tool( + "workspace_disk_read", + { + "workspace_id": "workspace-123", + "path": "note.txt", + "max_bytes": 4096, + }, + ) + ) + return stopped, started, exported, listed, read + + stopped, started, exported, listed, read = asyncio.run(_run()) + assert stopped["state"] == "stopped" + assert started["state"] == "started" + assert exported["output_path"] == "/tmp/workspace.ext4" + assert listed["entries"] == [] + assert read["content"] == "" + assert calls == [ + ("stop_workspace", {"workspace_id": "workspace-123"}), + ("start_workspace", {"workspace_id": "workspace-123"}), + ( + "export_workspace_disk", + { + "workspace_id": "workspace-123", + "output_path": "/tmp/workspace.ext4", + }, + ), + ( + "list_workspace_disk", + { + "workspace_id": "workspace-123", + "path": "/workspace/src", + "recursive": True, + }, + ), + ( + "read_workspace_disk", + { + "workspace_id": "workspace-123", + "path": "note.txt", + "max_bytes": 4096, + }, + ), + ] + + +def test_pyro_create_server_workspace_status_shell_and_service_delegate() -> None: + calls: list[tuple[str, dict[str, Any]]] = [] + + class StubManager: + def status_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("status_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "state": "started"} + + def logs_workspace(self, workspace_id: str) -> dict[str, Any]: + calls.append(("logs_workspace", {"workspace_id": workspace_id})) + return {"workspace_id": workspace_id, "count": 0, "entries": []} + + def open_shell( + self, + workspace_id: str, + *, + cwd: str = "/workspace", + cols: int = 120, + rows: int = 30, + secret_env: dict[str, str] | None = None, + ) -> dict[str, Any]: + calls.append( + ( + "open_shell", + { + "workspace_id": workspace_id, + "cwd": cwd, + "cols": cols, + "rows": rows, + "secret_env": secret_env, + }, + ) + ) + return {"workspace_id": workspace_id, "shell_id": "shell-1", "state": "running"} + + def read_shell( + self, + workspace_id: str, + shell_id: str, + *, + cursor: int = 0, + max_chars: int = 65536, + ) -> dict[str, Any]: + calls.append( + ( + "read_shell", + { + "workspace_id": workspace_id, + "shell_id": shell_id, + "cursor": cursor, + "max_chars": max_chars, + }, + ) + ) + return {"workspace_id": workspace_id, "shell_id": shell_id, "output": ""} + + def write_shell( + self, + workspace_id: str, + shell_id: str, + *, + input_text: str, + append_newline: bool = True, + ) -> dict[str, Any]: + calls.append( + ( + "write_shell", + { + "workspace_id": workspace_id, + "shell_id": shell_id, + "input_text": input_text, + "append_newline": append_newline, + }, + ) + ) + return { + "workspace_id": workspace_id, + "shell_id": shell_id, + "input_length": len(input_text), + } + + def signal_shell( + self, + workspace_id: str, + shell_id: str, + *, + signal_name: str = "INT", + ) -> dict[str, Any]: + calls.append( + ( + "signal_shell", + { + "workspace_id": workspace_id, + "shell_id": shell_id, + "signal_name": signal_name, + }, + ) + ) + return {"workspace_id": workspace_id, "shell_id": shell_id, "signal": signal_name} + + def close_shell(self, workspace_id: str, shell_id: str) -> dict[str, Any]: + calls.append( + ("close_shell", {"workspace_id": workspace_id, "shell_id": shell_id}) + ) + return {"workspace_id": workspace_id, "shell_id": shell_id, "closed": True} + + def start_service( + self, + workspace_id: str, + service_name: str, + **kwargs: Any, + ) -> dict[str, Any]: + calls.append( + ( + "start_service", + { + "workspace_id": workspace_id, + "service_name": service_name, + **kwargs, + }, + ) + ) + return {"workspace_id": workspace_id, "service_name": service_name, "state": "running"} + + pyro = Pyro(manager=cast(Any, StubManager())) + + def _extract_structured(raw_result: object) -> dict[str, Any]: + if not isinstance(raw_result, tuple) or len(raw_result) != 2: + raise TypeError("unexpected call_tool result shape") + _, structured = raw_result + if not isinstance(structured, dict): + raise TypeError("expected structured dictionary result") + return cast(dict[str, Any], structured) + + async def _run() -> tuple[dict[str, Any], ...]: + server = pyro.create_server() + status = _extract_structured( + await server.call_tool("workspace_status", {"workspace_id": "workspace-123"}) + ) + logs = _extract_structured( + await server.call_tool("workspace_logs", {"workspace_id": "workspace-123"}) + ) + opened = _extract_structured( + await server.call_tool( + "shell_open", + { + "workspace_id": "workspace-123", + "cwd": "/workspace/src", + "cols": 100, + "rows": 20, + "secret_env": {"TOKEN": "API_TOKEN"}, + }, + ) + ) + read = _extract_structured( + await server.call_tool( + "shell_read", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "cursor": 5, + "max_chars": 1024, + }, + ) + ) + wrote = _extract_structured( + await server.call_tool( + "shell_write", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "input": "pwd", + "append_newline": False, + }, + ) + ) + signaled = _extract_structured( + await server.call_tool( + "shell_signal", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "signal_name": "TERM", + }, + ) + ) + closed = _extract_structured( + await server.call_tool( + "shell_close", + {"workspace_id": "workspace-123", "shell_id": "shell-1"}, + ) + ) + file_service = _extract_structured( + await server.call_tool( + "service_start", + { + "workspace_id": "workspace-123", + "service_name": "file", + "command": "run-file", + "ready_file": ".ready", + }, + ) + ) + tcp_service = _extract_structured( + await server.call_tool( + "service_start", + { + "workspace_id": "workspace-123", + "service_name": "tcp", + "command": "run-tcp", + "ready_tcp": "127.0.0.1:8080", + }, + ) + ) + http_service = _extract_structured( + await server.call_tool( + "service_start", + { + "workspace_id": "workspace-123", + "service_name": "http", + "command": "run-http", + "ready_http": "http://127.0.0.1:8080/", + }, + ) + ) + command_service = _extract_structured( + await server.call_tool( + "service_start", + { + "workspace_id": "workspace-123", + "service_name": "command", + "command": "run-command", + "ready_command": "test -f .ready", + }, + ) + ) + return ( + status, + logs, + opened, + read, + wrote, + signaled, + closed, + file_service, + tcp_service, + http_service, + command_service, + ) + + results = asyncio.run(_run()) + assert results[0]["state"] == "started" + assert results[1]["count"] == 0 + assert results[2]["shell_id"] == "shell-1" + assert results[6]["closed"] is True + assert results[7]["state"] == "running" + assert results[10]["state"] == "running" + assert calls == [ + ("status_workspace", {"workspace_id": "workspace-123"}), + ("logs_workspace", {"workspace_id": "workspace-123"}), + ( + "open_shell", + { + "workspace_id": "workspace-123", + "cwd": "/workspace/src", + "cols": 100, + "rows": 20, + "secret_env": {"TOKEN": "API_TOKEN"}, + }, + ), + ( + "read_shell", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "cursor": 5, + "max_chars": 1024, + }, + ), + ( + "write_shell", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "input_text": "pwd", + "append_newline": False, + }, + ), + ( + "signal_shell", + { + "workspace_id": "workspace-123", + "shell_id": "shell-1", + "signal_name": "TERM", + }, + ), + ("close_shell", {"workspace_id": "workspace-123", "shell_id": "shell-1"}), + ( + "start_service", + { + "workspace_id": "workspace-123", + "service_name": "file", + "command": "run-file", + "cwd": "/workspace", + "readiness": {"type": "file", "path": ".ready"}, + "ready_timeout_seconds": 30, + "ready_interval_ms": 500, + "secret_env": None, + "published_ports": None, + }, + ), + ( + "start_service", + { + "workspace_id": "workspace-123", + "service_name": "tcp", + "command": "run-tcp", + "cwd": "/workspace", + "readiness": {"type": "tcp", "address": "127.0.0.1:8080"}, + "ready_timeout_seconds": 30, + "ready_interval_ms": 500, + "secret_env": None, + "published_ports": None, + }, + ), + ( + "start_service", + { + "workspace_id": "workspace-123", + "service_name": "http", + "command": "run-http", + "cwd": "/workspace", + "readiness": {"type": "http", "url": "http://127.0.0.1:8080/"}, + "ready_timeout_seconds": 30, + "ready_interval_ms": 500, + "secret_env": None, + "published_ports": None, + }, + ), + ( + "start_service", + { + "workspace_id": "workspace-123", + "service_name": "command", + "command": "run-command", + "cwd": "/workspace", + "readiness": {"type": "command", "command": "test -f .ready"}, + "ready_timeout_seconds": 30, + "ready_interval_ms": 500, + "secret_env": None, + "published_ports": None, + }, + ), + ] diff --git a/tests/test_cli.py b/tests/test_cli.py index 609abb2..872c5a0 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -72,6 +72,10 @@ def test_cli_subcommand_help_includes_examples_and_guidance() -> None: assert "pyro workspace exec WORKSPACE_ID" in workspace_help assert "pyro workspace diff WORKSPACE_ID" in workspace_help assert "pyro workspace export WORKSPACE_ID src/note.txt --output ./note.txt" in workspace_help + assert "pyro workspace stop WORKSPACE_ID" in workspace_help + assert "pyro workspace disk list WORKSPACE_ID" in workspace_help + assert "pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4" in workspace_help + assert "pyro workspace start WORKSPACE_ID" in workspace_help assert "pyro workspace snapshot create WORKSPACE_ID checkpoint" in workspace_help assert "pyro workspace reset WORKSPACE_ID --snapshot checkpoint" in workspace_help assert "pyro workspace shell open WORKSPACE_ID" in workspace_help @@ -112,6 +116,37 @@ def test_cli_subcommand_help_includes_examples_and_guidance() -> None: assert "--output" in workspace_export_help assert "Export one file or directory from `/workspace`" in workspace_export_help + workspace_stop_help = _subparser_choice( + _subparser_choice(parser, "workspace"), "stop" + ).format_help() + assert "Stop the backing sandbox" in workspace_stop_help + + workspace_start_help = _subparser_choice( + _subparser_choice(parser, "workspace"), "start" + ).format_help() + assert "previously stopped workspace" in workspace_start_help + + workspace_disk_help = _subparser_choice( + _subparser_choice(parser, "workspace"), "disk" + ).format_help() + assert "secondary stopped-workspace disk tools" in workspace_disk_help + assert "pyro workspace disk read WORKSPACE_ID note.txt" in workspace_disk_help + + workspace_disk_export_help = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "export" + ).format_help() + assert "--output" in workspace_disk_export_help + + workspace_disk_list_help = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "list" + ).format_help() + assert "--recursive" in workspace_disk_list_help + + workspace_disk_read_help = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "read" + ).format_help() + assert "--max-bytes" in workspace_disk_read_help + workspace_diff_help = _subparser_choice( _subparser_choice(parser, "workspace"), "diff" ).format_help() @@ -647,6 +682,193 @@ def test_cli_workspace_export_prints_human_output( assert "artifact_type=file" in output +def test_cli_workspace_stop_and_start_print_human_output( + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + class StubPyro: + def stop_workspace(self, workspace_id: str) -> dict[str, Any]: + assert workspace_id == "workspace-123" + return { + "workspace_id": workspace_id, + "environment": "debian:12", + "state": "stopped", + "workspace_path": "/workspace", + "network_policy": "off", + "execution_mode": "guest_vsock", + "vcpu_count": 1, + "mem_mib": 1024, + "command_count": 2, + "reset_count": 0, + "service_count": 0, + "running_service_count": 0, + } + + def start_workspace(self, workspace_id: str) -> dict[str, Any]: + assert workspace_id == "workspace-123" + return { + "workspace_id": workspace_id, + "environment": "debian:12", + "state": "started", + "workspace_path": "/workspace", + "network_policy": "off", + "execution_mode": "guest_vsock", + "vcpu_count": 1, + "mem_mib": 1024, + "command_count": 2, + "reset_count": 0, + "service_count": 0, + "running_service_count": 0, + } + + class StopParser: + def parse_args(self) -> argparse.Namespace: + return argparse.Namespace( + command="workspace", + workspace_command="stop", + workspace_id="workspace-123", + json=False, + ) + + class StartParser: + def parse_args(self) -> argparse.Namespace: + return argparse.Namespace( + command="workspace", + workspace_command="start", + workspace_id="workspace-123", + json=False, + ) + + monkeypatch.setattr(cli, "Pyro", StubPyro) + monkeypatch.setattr(cli, "_build_parser", lambda: StopParser()) + cli.main() + stopped_output = capsys.readouterr().out + assert "Stopped workspace ID: workspace-123" in stopped_output + + monkeypatch.setattr(cli, "_build_parser", lambda: StartParser()) + cli.main() + started_output = capsys.readouterr().out + assert "Started workspace ID: workspace-123" in started_output + + +def test_cli_workspace_disk_commands_print_human_and_json( + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], +) -> None: + class StubPyro: + def export_workspace_disk( + self, + workspace_id: str, + *, + output_path: str, + ) -> dict[str, Any]: + assert workspace_id == "workspace-123" + assert output_path == "./workspace.ext4" + return { + "workspace_id": workspace_id, + "output_path": "/tmp/workspace.ext4", + "disk_format": "ext4", + "bytes_written": 8192, + } + + def list_workspace_disk( + self, + workspace_id: str, + *, + path: str, + recursive: bool, + ) -> dict[str, Any]: + assert workspace_id == "workspace-123" + assert path == "/workspace" + assert recursive is True + return { + "workspace_id": workspace_id, + "path": path, + "recursive": recursive, + "entries": [ + { + "path": "/workspace/note.txt", + "artifact_type": "file", + "size_bytes": 6, + "link_target": None, + } + ], + } + + def read_workspace_disk( + self, + workspace_id: str, + path: str, + *, + max_bytes: int, + ) -> dict[str, Any]: + assert workspace_id == "workspace-123" + assert path == "note.txt" + assert max_bytes == 4096 + return { + "workspace_id": workspace_id, + "path": "/workspace/note.txt", + "size_bytes": 6, + "max_bytes": max_bytes, + "content": "hello\n", + "truncated": False, + } + + class ExportParser: + def parse_args(self) -> argparse.Namespace: + return argparse.Namespace( + command="workspace", + workspace_command="disk", + workspace_disk_command="export", + workspace_id="workspace-123", + output="./workspace.ext4", + json=False, + ) + + class ListParser: + def parse_args(self) -> argparse.Namespace: + return argparse.Namespace( + command="workspace", + workspace_command="disk", + workspace_disk_command="list", + workspace_id="workspace-123", + path="/workspace", + recursive=True, + json=False, + ) + + class ReadParser: + def parse_args(self) -> argparse.Namespace: + return argparse.Namespace( + command="workspace", + workspace_command="disk", + workspace_disk_command="read", + workspace_id="workspace-123", + path="note.txt", + max_bytes=4096, + json=True, + ) + + monkeypatch.setattr(cli, "Pyro", StubPyro) + + monkeypatch.setattr(cli, "_build_parser", lambda: ExportParser()) + cli.main() + export_output = capsys.readouterr().out + assert "[workspace-disk-export] workspace_id=workspace-123" in export_output + + monkeypatch.setattr(cli, "_build_parser", lambda: ListParser()) + cli.main() + list_output = capsys.readouterr().out + assert "Workspace disk path: /workspace" in list_output + assert "/workspace/note.txt [file]" in list_output + + monkeypatch.setattr(cli, "_build_parser", lambda: ReadParser()) + cli.main() + read_payload = json.loads(capsys.readouterr().out) + assert read_payload["path"] == "/workspace/note.txt" + assert read_payload["content"] == "hello\n" + + def test_cli_workspace_diff_prints_human_output( monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], diff --git a/tests/test_doctor.py b/tests/test_doctor.py index 88ab026..9629771 100644 --- a/tests/test_doctor.py +++ b/tests/test_doctor.py @@ -6,6 +6,7 @@ import json import pytest import pyro_mcp.doctor as doctor_module +from pyro_mcp.runtime import DEFAULT_PLATFORM def test_doctor_main_prints_json( @@ -25,3 +26,9 @@ def test_doctor_main_prints_json( doctor_module.main() output = json.loads(capsys.readouterr().out) assert output["runtime_ok"] is True + + +def test_doctor_build_parser_defaults_platform() -> None: + parser = doctor_module._build_parser() + args = parser.parse_args([]) + assert args.platform == DEFAULT_PLATFORM diff --git a/tests/test_package_surface.py b/tests/test_package_surface.py new file mode 100644 index 0000000..533a87a --- /dev/null +++ b/tests/test_package_surface.py @@ -0,0 +1,67 @@ +from __future__ import annotations + +from importlib.metadata import PackageNotFoundError +from typing import Any, cast + +import pyro_mcp as package_module + + +def test_resolve_version_prefers_pyproject_version(monkeypatch: Any) -> None: + monkeypatch.setattr(package_module, "version", lambda _name: "9.9.9") + assert package_module._resolve_version() == package_module.__version__ # noqa: SLF001 + + +def test_resolve_version_falls_back_to_unknown_without_metadata(monkeypatch: Any) -> None: + class _FakePyprojectPath: + def exists(self) -> bool: + return False + + class _FakeResolvedPath: + @property + def parents(self) -> dict[int, Any]: + return {2: self} + + def __truediv__(self, _other: str) -> _FakePyprojectPath: + return _FakePyprojectPath() + + class _FakePathFactory: + def __init__(self, _value: str) -> None: + return None + + def resolve(self) -> _FakeResolvedPath: + return _FakeResolvedPath() + + monkeypatch.setattr( + package_module, + "version", + lambda _name: (_ for _ in ()).throw(PackageNotFoundError()), + ) + monkeypatch.setattr(package_module, "Path", cast(Any, _FakePathFactory)) + + assert package_module._resolve_version() == "0+unknown" # noqa: SLF001 + + +def test_resolve_version_falls_back_to_installed_version(monkeypatch: Any) -> None: + class _FakePyprojectPath: + def exists(self) -> bool: + return False + + class _FakeResolvedPath: + @property + def parents(self) -> dict[int, Any]: + return {2: self} + + def __truediv__(self, _other: str) -> _FakePyprojectPath: + return _FakePyprojectPath() + + class _FakePathFactory: + def __init__(self, _value: str) -> None: + return None + + def resolve(self) -> _FakeResolvedPath: + return _FakeResolvedPath() + + monkeypatch.setattr(package_module, "version", lambda _name: "9.9.9") + monkeypatch.setattr(package_module, "Path", cast(Any, _FakePathFactory)) + + assert package_module._resolve_version() == "9.9.9" # noqa: SLF001 diff --git a/tests/test_public_contract.py b/tests/test_public_contract.py index 6f22889..ab5ad98 100644 --- a/tests/test_public_contract.py +++ b/tests/test_public_contract.py @@ -19,6 +19,9 @@ from pyro_mcp.contract import ( PUBLIC_CLI_RUN_FLAGS, PUBLIC_CLI_WORKSPACE_CREATE_FLAGS, PUBLIC_CLI_WORKSPACE_DIFF_FLAGS, + PUBLIC_CLI_WORKSPACE_DISK_EXPORT_FLAGS, + PUBLIC_CLI_WORKSPACE_DISK_LIST_FLAGS, + PUBLIC_CLI_WORKSPACE_DISK_READ_FLAGS, PUBLIC_CLI_WORKSPACE_EXEC_FLAGS, PUBLIC_CLI_WORKSPACE_EXPORT_FLAGS, PUBLIC_CLI_WORKSPACE_RESET_FLAGS, @@ -38,6 +41,8 @@ from pyro_mcp.contract import ( PUBLIC_CLI_WORKSPACE_SNAPSHOT_DELETE_FLAGS, PUBLIC_CLI_WORKSPACE_SNAPSHOT_LIST_FLAGS, PUBLIC_CLI_WORKSPACE_SNAPSHOT_SUBCOMMANDS, + PUBLIC_CLI_WORKSPACE_START_FLAGS, + PUBLIC_CLI_WORKSPACE_STOP_FLAGS, PUBLIC_CLI_WORKSPACE_SUBCOMMANDS, PUBLIC_CLI_WORKSPACE_SYNC_PUSH_FLAGS, PUBLIC_CLI_WORKSPACE_SYNC_SUBCOMMANDS, @@ -116,6 +121,26 @@ def test_public_cli_help_lists_commands_and_run_flags() -> None: ).format_help() for flag in PUBLIC_CLI_WORKSPACE_EXPORT_FLAGS: assert flag in workspace_export_help_text + workspace_disk_help_text = _subparser_choice( + _subparser_choice(parser, "workspace"), "disk" + ).format_help() + for subcommand_name in ("export", "list", "read"): + assert subcommand_name in workspace_disk_help_text + workspace_disk_export_help_text = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "export" + ).format_help() + for flag in PUBLIC_CLI_WORKSPACE_DISK_EXPORT_FLAGS: + assert flag in workspace_disk_export_help_text + workspace_disk_list_help_text = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "list" + ).format_help() + for flag in PUBLIC_CLI_WORKSPACE_DISK_LIST_FLAGS: + assert flag in workspace_disk_list_help_text + workspace_disk_read_help_text = _subparser_choice( + _subparser_choice(_subparser_choice(parser, "workspace"), "disk"), "read" + ).format_help() + for flag in PUBLIC_CLI_WORKSPACE_DISK_READ_FLAGS: + assert flag in workspace_disk_read_help_text workspace_diff_help_text = _subparser_choice( _subparser_choice(parser, "workspace"), "diff" ).format_help() @@ -150,6 +175,16 @@ def test_public_cli_help_lists_commands_and_run_flags() -> None: ).format_help() for flag in PUBLIC_CLI_WORKSPACE_RESET_FLAGS: assert flag in workspace_reset_help_text + workspace_start_help_text = _subparser_choice( + _subparser_choice(parser, "workspace"), "start" + ).format_help() + for flag in PUBLIC_CLI_WORKSPACE_START_FLAGS: + assert flag in workspace_start_help_text + workspace_stop_help_text = _subparser_choice( + _subparser_choice(parser, "workspace"), "stop" + ).format_help() + for flag in PUBLIC_CLI_WORKSPACE_STOP_FLAGS: + assert flag in workspace_stop_help_text workspace_shell_help_text = _subparser_choice( _subparser_choice(parser, "workspace"), "shell", diff --git a/tests/test_runtime_boot_check.py b/tests/test_runtime_boot_check.py index 0733e0a..6e55134 100644 --- a/tests/test_runtime_boot_check.py +++ b/tests/test_runtime_boot_check.py @@ -1,6 +1,8 @@ from __future__ import annotations -from pyro_mcp.runtime_boot_check import _classify_result +import pytest + +from pyro_mcp.runtime_boot_check import _classify_result, run_boot_check def test_classify_result_reports_kernel_panic() -> None: @@ -19,3 +21,32 @@ def test_classify_result_reports_success_when_vm_stays_alive() -> None: vm_alive=True, ) assert reason is None + + +def test_classify_result_reports_logger_failure_and_early_exit() -> None: + logger_reason = _classify_result( + firecracker_log="Successfully started microvm", + serial_log="Could not initialize logger", + vm_alive=False, + ) + early_exit_reason = _classify_result( + firecracker_log="partial log", + serial_log="boot log", + vm_alive=False, + ) + assert logger_reason == "firecracker logger initialization failed" + assert early_exit_reason == "firecracker did not fully start the microVM" + + +def test_classify_result_reports_boot_window_exit_after_start() -> None: + reason = _classify_result( + firecracker_log="Successfully started microvm", + serial_log="boot log", + vm_alive=False, + ) + assert reason == "microVM exited before boot validation window elapsed" + + +def test_run_boot_check_requires_positive_wait_seconds() -> None: + with pytest.raises(ValueError, match="wait_seconds must be positive"): + run_boot_check(wait_seconds=0) diff --git a/tests/test_server.py b/tests/test_server.py index f9eac6d..5cfa044 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -32,8 +32,13 @@ def test_create_server_registers_vm_tools(tmp_path: Path) -> None: assert "vm_run" in tool_names assert "vm_status" in tool_names assert "workspace_create" in tool_names + assert "workspace_start" in tool_names + assert "workspace_stop" in tool_names assert "workspace_diff" in tool_names assert "workspace_export" in tool_names + assert "workspace_disk_export" in tool_names + assert "workspace_disk_list" in tool_names + assert "workspace_disk_read" in tool_names assert "workspace_logs" in tool_names assert "workspace_sync_push" in tool_names assert "shell_open" in tool_names diff --git a/tests/test_vm_manager.py b/tests/test_vm_manager.py index 8e31754..0613806 100644 --- a/tests/test_vm_manager.py +++ b/tests/test_vm_manager.py @@ -18,6 +18,55 @@ from pyro_mcp.vm_manager import VmManager from pyro_mcp.vm_network import NetworkConfig, TapNetworkManager +def _run_debugfs_write(rootfs_image: Path, command: str) -> None: + proc = subprocess.run( # noqa: S603 + ["debugfs", "-w", "-R", command, str(rootfs_image)], + text=True, + capture_output=True, + check=False, + ) + if proc.returncode != 0: + message = proc.stderr.strip() or proc.stdout.strip() or command + raise RuntimeError(message) + + +def _create_stopped_workspace_rootfs(tmp_path: Path) -> Path: + rootfs_image = tmp_path / "workspace-rootfs.ext4" + with rootfs_image.open("wb") as handle: + handle.truncate(16 * 1024 * 1024) + proc = subprocess.run( # noqa: S603 + ["mkfs.ext4", "-F", str(rootfs_image)], + text=True, + capture_output=True, + check=False, + ) + if proc.returncode != 0: + message = proc.stderr.strip() or proc.stdout.strip() or "mkfs.ext4 failed" + raise RuntimeError(message) + for directory in ( + "/workspace", + "/workspace/src", + "/run", + "/run/pyro-secrets", + "/run/pyro-services", + ): + _run_debugfs_write(rootfs_image, f"mkdir {directory}") + note_path = tmp_path / "note.txt" + note_path.write_text("hello from disk\n", encoding="utf-8") + child_path = tmp_path / "child.txt" + child_path.write_text("nested child\n", encoding="utf-8") + secret_path = tmp_path / "secret.txt" + secret_path.write_text("super-secret\n", encoding="utf-8") + service_path = tmp_path / "service.log" + service_path.write_text("service runtime\n", encoding="utf-8") + _run_debugfs_write(rootfs_image, f"write {note_path} /workspace/note.txt") + _run_debugfs_write(rootfs_image, f"write {child_path} /workspace/src/child.txt") + _run_debugfs_write(rootfs_image, "symlink /workspace/link note.txt") + _run_debugfs_write(rootfs_image, f"write {secret_path} /run/pyro-secrets/TOKEN") + _run_debugfs_write(rootfs_image, f"write {service_path} /run/pyro-services/app.log") + return rootfs_image + + def test_vm_manager_lifecycle_and_auto_cleanup(tmp_path: Path) -> None: manager = VmManager( backend_name="mock", @@ -1129,6 +1178,80 @@ def test_vm_manager_firecracker_backend_path( assert manager._backend_name == "firecracker" # noqa: SLF001 +def test_firecracker_backend_start_removes_stale_socket_files( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + backend = cast(Any, object.__new__(vm_manager_module.FirecrackerBackend)) + backend._environment_store = object() # noqa: SLF001 + backend._firecracker_bin = tmp_path / "firecracker" # noqa: SLF001 + backend._jailer_bin = tmp_path / "jailer" # noqa: SLF001 + backend._runtime_capabilities = RuntimeCapabilities( # noqa: SLF001 + supports_vm_boot=True, + supports_guest_exec=True, + supports_guest_network=False, + reason=None, + ) + backend._network_manager = TapNetworkManager(enabled=False) # noqa: SLF001 + backend._guest_exec_client = None # noqa: SLF001 + backend._processes = {} # noqa: SLF001 + + backend._firecracker_bin.write_text("fc", encoding="utf-8") # noqa: SLF001 + backend._jailer_bin.write_text("jailer", encoding="utf-8") # noqa: SLF001 + kernel_image = tmp_path / "vmlinux" + kernel_image.write_text("kernel", encoding="utf-8") + rootfs_image = tmp_path / "rootfs.ext4" + rootfs_image.write_bytes(b"rootfs") + + workdir = tmp_path / "runtime" + workdir.mkdir() + firecracker_socket = workdir / "firecracker.sock" + vsock_socket = workdir / "vsock.sock" + firecracker_socket.write_text("stale firecracker socket", encoding="utf-8") + vsock_socket.write_text("stale vsock socket", encoding="utf-8") + + class DummyPopen: + def __init__(self, *args: Any, **kwargs: Any) -> None: + del args, kwargs + self.pid = 4242 + + def poll(self) -> None: + return None + + monkeypatch.setattr( + cast(Any, vm_manager_module).subprocess, + "run", + lambda *args, **kwargs: subprocess.CompletedProcess( # noqa: ARG005 + args=args[0], + returncode=0, + stdout="Firecracker v1.0.0\n", + stderr="", + ), + ) + monkeypatch.setattr(cast(Any, vm_manager_module).subprocess, "Popen", DummyPopen) + + instance = vm_manager_module.VmInstance( + vm_id="abcd1234", + environment="debian:12", + vcpu_count=1, + mem_mib=512, + ttl_seconds=600, + created_at=time.time(), + expires_at=time.time() + 600, + workdir=workdir, + metadata={ + "kernel_image": str(kernel_image), + "rootfs_image": str(rootfs_image), + }, + ) + + backend.start(instance) + + assert instance.firecracker_pid == 4242 + assert not firecracker_socket.exists() + assert not vsock_socket.exists() + + def test_vm_manager_fails_closed_without_host_compat_opt_in(tmp_path: Path) -> None: manager = VmManager( backend_name="mock", @@ -2691,3 +2814,181 @@ def test_workspace_secrets_require_guest_exec_on_firecracker_runtime( allow_host_compat=True, secrets=[{"name": "TOKEN", "value": "expected"}], ) + + +def test_workspace_stop_and_start_preserve_logs_and_clear_live_state(tmp_path: Path) -> None: + manager = VmManager( + backend_name="mock", + base_dir=tmp_path / "vms", + network_manager=TapNetworkManager(enabled=False), + ) + seed_dir = tmp_path / "seed" + seed_dir.mkdir() + (seed_dir / "note.txt").write_text("hello from seed\n", encoding="utf-8") + + created = manager.create_workspace( + environment="debian:12-base", + allow_host_compat=True, + seed_path=seed_dir, + ) + workspace_id = str(created["workspace_id"]) + manager.exec_workspace(workspace_id, command="cat note.txt", timeout_seconds=30) + shell = manager.open_shell(workspace_id) + shell_id = str(shell["shell_id"]) + started_service = manager.start_service( + workspace_id, + "app", + command='sh -lc \'touch .ready && trap "exit 0" TERM; while true; do sleep 60; done\'', + readiness={"type": "file", "path": ".ready"}, + ) + assert started_service["state"] == "running" + + stopped = manager.stop_workspace(workspace_id) + assert stopped["state"] == "stopped" + assert stopped["command_count"] == 1 + assert stopped["service_count"] == 0 + assert stopped["running_service_count"] == 0 + assert manager.logs_workspace(workspace_id)["count"] == 1 + with pytest.raises(RuntimeError, match="must be in 'started' state"): + manager.read_shell(workspace_id, shell_id, cursor=0, max_chars=1024) + + restarted = manager.start_workspace(workspace_id) + assert restarted["state"] == "started" + assert restarted["command_count"] == 1 + assert restarted["service_count"] == 0 + rerun = manager.exec_workspace(workspace_id, command="cat note.txt", timeout_seconds=30) + assert rerun["stdout"] == "hello from seed\n" + + +def test_workspace_stop_flushes_guest_filesystem_before_stopping( + tmp_path: Path, +) -> None: + manager = VmManager( + backend_name="mock", + base_dir=tmp_path / "vms", + network_manager=TapNetworkManager(enabled=False), + ) + created = manager.create_workspace( + environment="debian:12-base", + allow_host_compat=True, + ) + workspace_id = str(created["workspace_id"]) + workspace_path = tmp_path / "vms" / "workspaces" / workspace_id / "workspace.json" + payload = json.loads(workspace_path.read_text(encoding="utf-8")) + payload["state"] = "started" + payload["firecracker_pid"] = os.getpid() + payload["metadata"]["execution_mode"] = "guest_vsock" + payload["metadata"]["rootfs_image"] = str(_create_stopped_workspace_rootfs(tmp_path)) + workspace_path.write_text(json.dumps(payload, indent=2, sort_keys=True), encoding="utf-8") + + calls: list[tuple[str, str]] = [] + + class StubBackend: + def exec( + self, + instance: Any, + command: str, + timeout_seconds: int, + *, + workdir: Path | None = None, + env: dict[str, str] | None = None, + ) -> vm_manager_module.VmExecResult: + del instance, timeout_seconds, workdir, env + calls.append(("exec", command)) + return vm_manager_module.VmExecResult( + stdout="", + stderr="", + exit_code=0, + duration_ms=1, + ) + + def stop(self, instance: Any) -> None: + del instance + calls.append(("stop", "instance")) + + manager._backend = StubBackend() # type: ignore[assignment] # noqa: SLF001 + manager._backend_name = "firecracker" # noqa: SLF001 + manager._runtime_capabilities = RuntimeCapabilities( # noqa: SLF001 + supports_vm_boot=True, + supports_guest_exec=True, + supports_guest_network=False, + reason=None, + ) + + stopped = manager.stop_workspace(workspace_id) + + assert calls == [("exec", "sync"), ("stop", "instance")] + assert stopped["state"] == "stopped" + + +def test_workspace_disk_operations_scrub_runtime_only_paths_and_export( + tmp_path: Path, +) -> None: + manager = VmManager( + backend_name="mock", + base_dir=tmp_path / "vms", + network_manager=TapNetworkManager(enabled=False), + ) + rootfs_image = _create_stopped_workspace_rootfs(tmp_path) + workspace_id = "workspace-disk-123" + workspace = vm_manager_module.WorkspaceRecord( + workspace_id=workspace_id, + environment="debian:12-base", + vcpu_count=1, + mem_mib=512, + ttl_seconds=600, + created_at=time.time(), + expires_at=time.time() + 600, + state="stopped", + network_policy="off", + allow_host_compat=False, + metadata={ + "execution_mode": "guest_vsock", + "rootfs_image": str(rootfs_image), + "workspace_path": "/workspace", + }, + ) + manager._save_workspace_locked(workspace) # noqa: SLF001 + + listed = manager.list_workspace_disk(workspace_id, path="/workspace", recursive=True) + assert listed["path"] == "/workspace" + listed_paths = {entry["path"] for entry in listed["entries"]} + assert "/workspace/note.txt" in listed_paths + assert "/workspace/src/child.txt" in listed_paths + assert "/workspace/link" in listed_paths + + read_payload = manager.read_workspace_disk(workspace_id, path="note.txt", max_bytes=4096) + assert read_payload["content"] == "hello from disk\n" + assert read_payload["truncated"] is False + + run_listing = manager.list_workspace_disk(workspace_id, path="/run", recursive=True) + run_paths = {entry["path"] for entry in run_listing["entries"]} + assert "/run/pyro-secrets" not in run_paths + assert "/run/pyro-services" not in run_paths + + exported_path = tmp_path / "workspace-copy.ext4" + exported = manager.export_workspace_disk(workspace_id, output_path=exported_path) + assert exported["disk_format"] == "ext4" + assert exported_path.exists() + assert exported_path.stat().st_size == int(exported["bytes_written"]) + + +def test_workspace_disk_operations_reject_host_compat_workspaces(tmp_path: Path) -> None: + manager = VmManager( + backend_name="mock", + base_dir=tmp_path / "vms", + network_manager=TapNetworkManager(enabled=False), + ) + created = manager.create_workspace( + environment="debian:12-base", + allow_host_compat=True, + ) + workspace_id = str(created["workspace_id"]) + manager.stop_workspace(workspace_id) + + with pytest.raises(RuntimeError, match="host_compat workspaces"): + manager.export_workspace_disk(workspace_id, output_path=tmp_path / "workspace.ext4") + with pytest.raises(RuntimeError, match="host_compat workspaces"): + manager.list_workspace_disk(workspace_id) + with pytest.raises(RuntimeError, match="host_compat workspaces"): + manager.read_workspace_disk(workspace_id, path="note.txt") diff --git a/tests/test_workspace_disk.py b/tests/test_workspace_disk.py new file mode 100644 index 0000000..6f1e3de --- /dev/null +++ b/tests/test_workspace_disk.py @@ -0,0 +1,258 @@ +from __future__ import annotations + +import subprocess +from pathlib import Path +from types import SimpleNamespace +from typing import Any, cast + +import pytest + +import pyro_mcp.workspace_disk as workspace_disk_module +from pyro_mcp.workspace_disk import ( + _artifact_type_from_mode, + _debugfs_ls_entries, + _debugfs_stat, + _run_debugfs, + export_workspace_disk_image, + list_workspace_disk, + read_workspace_disk_file, + scrub_workspace_runtime_paths, +) + + +def _run_debugfs_write(rootfs_image: Path, command: str) -> None: + proc = subprocess.run( # noqa: S603 + ["debugfs", "-w", "-R", command, str(rootfs_image)], + text=True, + capture_output=True, + check=False, + ) + if proc.returncode != 0: + message = proc.stderr.strip() or proc.stdout.strip() or command + raise RuntimeError(message) + + +def _create_rootfs_image(tmp_path: Path) -> Path: + rootfs_image = tmp_path / "workspace-rootfs.ext4" + with rootfs_image.open("wb") as handle: + handle.truncate(16 * 1024 * 1024) + proc = subprocess.run( # noqa: S603 + ["mkfs.ext4", "-F", str(rootfs_image)], + text=True, + capture_output=True, + check=False, + ) + if proc.returncode != 0: + message = proc.stderr.strip() or proc.stdout.strip() or "mkfs.ext4 failed" + raise RuntimeError(message) + for directory in ( + "/workspace", + "/workspace/src", + "/run", + "/run/pyro-secrets", + "/run/pyro-services", + ): + _run_debugfs_write(rootfs_image, f"mkdir {directory}") + note_path = tmp_path / "note.txt" + note_path.write_text("hello from disk\n", encoding="utf-8") + child_path = tmp_path / "child.txt" + child_path.write_text("nested child\n", encoding="utf-8") + secret_path = tmp_path / "secret.txt" + secret_path.write_text("super-secret\n", encoding="utf-8") + service_path = tmp_path / "service.log" + service_path.write_text("service runtime\n", encoding="utf-8") + _run_debugfs_write(rootfs_image, f"write {note_path} /workspace/note.txt") + _run_debugfs_write(rootfs_image, f"write {child_path} /workspace/src/child.txt") + _run_debugfs_write(rootfs_image, "symlink /workspace/link note.txt") + _run_debugfs_write(rootfs_image, f"write {secret_path} /run/pyro-secrets/TOKEN") + _run_debugfs_write(rootfs_image, f"write {service_path} /run/pyro-services/app.log") + return rootfs_image + + +def test_workspace_disk_list_read_export_and_scrub(tmp_path: Path) -> None: + rootfs_image = _create_rootfs_image(tmp_path) + + listing = list_workspace_disk(rootfs_image, guest_path="/workspace", recursive=True) + assert listing == [ + { + "path": "/workspace/link", + "artifact_type": "symlink", + "size_bytes": 8, + "link_target": "note.txt", + }, + { + "path": "/workspace/note.txt", + "artifact_type": "file", + "size_bytes": 16, + "link_target": None, + }, + { + "path": "/workspace/src", + "artifact_type": "directory", + "size_bytes": 0, + "link_target": None, + }, + { + "path": "/workspace/src/child.txt", + "artifact_type": "file", + "size_bytes": 13, + "link_target": None, + }, + ] + + single = list_workspace_disk(rootfs_image, guest_path="/workspace/note.txt", recursive=False) + assert single == [ + { + "path": "/workspace/note.txt", + "artifact_type": "file", + "size_bytes": 16, + "link_target": None, + } + ] + + read_payload = read_workspace_disk_file( + rootfs_image, + guest_path="/workspace/note.txt", + max_bytes=5, + ) + assert read_payload == { + "path": "/workspace/note.txt", + "size_bytes": 16, + "max_bytes": 5, + "content": "hello", + "truncated": True, + } + + output_path = tmp_path / "workspace.ext4" + exported = export_workspace_disk_image(rootfs_image, output_path=output_path) + assert exported["output_path"] == str(output_path) + assert exported["disk_format"] == "ext4" + assert int(exported["bytes_written"]) == output_path.stat().st_size + + scrub_workspace_runtime_paths(rootfs_image) + run_listing = list_workspace_disk(rootfs_image, guest_path="/run", recursive=True) + assert run_listing == [] + + +def test_workspace_disk_rejects_invalid_inputs(tmp_path: Path) -> None: + rootfs_image = _create_rootfs_image(tmp_path) + + with pytest.raises(RuntimeError, match="workspace disk path does not exist"): + list_workspace_disk(rootfs_image, guest_path="/missing", recursive=False) + + with pytest.raises(RuntimeError, match="workspace disk path does not exist"): + read_workspace_disk_file( + rootfs_image, + guest_path="/missing.txt", + max_bytes=4096, + ) + + with pytest.raises(RuntimeError, match="regular files"): + read_workspace_disk_file( + rootfs_image, + guest_path="/workspace/src", + max_bytes=4096, + ) + + with pytest.raises(ValueError, match="max_bytes must be positive"): + read_workspace_disk_file( + rootfs_image, + guest_path="/workspace/note.txt", + max_bytes=0, + ) + + output_path = tmp_path / "existing.ext4" + output_path.write_text("present\n", encoding="utf-8") + with pytest.raises(RuntimeError, match="output_path already exists"): + export_workspace_disk_image(rootfs_image, output_path=output_path) + + +def test_workspace_disk_internal_error_paths( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + rootfs_image = tmp_path / "dummy.ext4" + rootfs_image.write_bytes(b"rootfs") + + monkeypatch.setattr(cast(Any, workspace_disk_module).shutil, "which", lambda _name: None) + with pytest.raises(RuntimeError, match="debugfs is required"): + _run_debugfs(rootfs_image, "stat /workspace") + + monkeypatch.setattr( + cast(Any, workspace_disk_module).shutil, + "which", + lambda _name: "/usr/bin/debugfs", + ) + monkeypatch.setattr( + cast(Any, workspace_disk_module).subprocess, + "run", + lambda *args, **kwargs: SimpleNamespace( # noqa: ARG005 + returncode=1, + stdout="", + stderr="", + ), + ) + with pytest.raises(RuntimeError, match="debugfs command failed: stat /workspace"): + _run_debugfs(rootfs_image, "stat /workspace") + + assert _artifact_type_from_mode("00000") is None + + monkeypatch.setattr(workspace_disk_module, "_run_debugfs", lambda *_args, **_kwargs: "noise") + with pytest.raises(RuntimeError, match="failed to inspect workspace disk path"): + _debugfs_stat(rootfs_image, "/workspace/bad") + + monkeypatch.setattr( + workspace_disk_module, + "_run_debugfs", + lambda *_args, **_kwargs: "Type: fifo\nSize: 1\n", + ) + with pytest.raises(RuntimeError, match="unsupported workspace disk path type"): + _debugfs_stat(rootfs_image, "/workspace/fifo") + + monkeypatch.setattr( + workspace_disk_module, + "_run_debugfs", + lambda *_args, **_kwargs: "File not found by ext2_lookup", + ) + with pytest.raises(RuntimeError, match="workspace disk path does not exist"): + _debugfs_ls_entries(rootfs_image, "/workspace/missing") + + monkeypatch.setattr( + workspace_disk_module, + "_debugfs_stat", + lambda *_args, **_kwargs: workspace_disk_module._DebugfsStat( # noqa: SLF001 + path="/workspace", + artifact_type="directory", + size_bytes=0, + ), + ) + monkeypatch.setattr( + workspace_disk_module, + "_debugfs_ls_entries", + lambda *_args, **_kwargs: [ + workspace_disk_module._DebugfsDirEntry( # noqa: SLF001 + name="special", + path="/workspace/special", + artifact_type=None, + size_bytes=0, + ) + ], + ) + assert list_workspace_disk(rootfs_image, guest_path="/workspace", recursive=True) == [] + + monkeypatch.setattr( + workspace_disk_module, + "_debugfs_stat", + lambda *_args, **_kwargs: workspace_disk_module._DebugfsStat( # noqa: SLF001 + path="/workspace/note.txt", + artifact_type="file", + size_bytes=12, + ), + ) + monkeypatch.setattr(workspace_disk_module, "_run_debugfs", lambda *_args, **_kwargs: "") + with pytest.raises(RuntimeError, match="failed to dump workspace disk file"): + read_workspace_disk_file( + rootfs_image, + guest_path="/workspace/note.txt", + max_bytes=16, + ) diff --git a/uv.lock b/uv.lock index bf70717..681046b 100644 --- a/uv.lock +++ b/uv.lock @@ -706,7 +706,7 @@ crypto = [ [[package]] name = "pyro-mcp" -version = "3.0.0" +version = "3.1.0" source = { editable = "." } dependencies = [ { name = "mcp" },