pyro-mcp/README.md
Thales Maciel 287f6d100f Add stopped-workspace disk export and inspection
Finish the 3.1.0 secondary disk-tools milestone so stable workspaces can be
stopped, inspected offline, exported as raw ext4 images, and started again
without changing the primary workspace-first interaction model.

Add workspace stop/start plus workspace disk export/list/read across the CLI,
SDK, and MCP, backed by a new offline debugfs inspection helper and guest-only
validation. Scrub runtime-only guest state before disk inspection/export, and
fix the real guest reliability gaps by flushing the filesystem on stop and
removing stale Firecracker socket files before restart.

Update the docs, examples, changelog, and roadmap to mark 3.1.0 done, and
cover the new lifecycle/disk paths with API, CLI, manager, contract, and
package-surface tests.

Validation: uv lock; UV_CACHE_DIR=.uv-cache make check; UV_CACHE_DIR=.uv-cache
make dist-check; real guest-backed smoke for create, shell/service activity,
stop, workspace disk list/read/export, start, exec, and delete.
2026-03-12 20:57:16 -03:00

576 lines
22 KiB
Markdown

# pyro-mcp
`pyro-mcp` is a stable agent workspace product for one-shot commands and persistent work inside ephemeral Firecracker microVMs using curated Linux environments such as `debian:12`.
[![PyPI version](https://img.shields.io/pypi/v/pyro-mcp.svg)](https://pypi.org/project/pyro-mcp/)
This is for coding agents, MCP clients, and developers who want isolated command execution and stable disposable workspaces in ephemeral microVMs.
It exposes the same runtime in three public forms:
- the `pyro` CLI
- the Python SDK via `from pyro_mcp import Pyro`
- an MCP server so LLM clients can call VM tools directly
## Start Here
- Install: [docs/install.md](docs/install.md)
- Vision: [docs/vision.md](docs/vision.md)
- Workspace roadmap: [docs/roadmap/task-workspace-ga.md](docs/roadmap/task-workspace-ga.md)
- First run transcript: [docs/first-run.md](docs/first-run.md)
- Stable workspace walkthrough GIF: [docs/assets/workspace-first-run.gif](docs/assets/workspace-first-run.gif)
- Terminal walkthrough GIF: [docs/assets/first-run.gif](docs/assets/first-run.gif)
- PyPI package: [pypi.org/project/pyro-mcp](https://pypi.org/project/pyro-mcp/)
- What's new in 3.1.0: [CHANGELOG.md#310](CHANGELOG.md#310)
- Host requirements: [docs/host-requirements.md](docs/host-requirements.md)
- Integration targets: [docs/integrations.md](docs/integrations.md)
- Public contract: [docs/public-contract.md](docs/public-contract.md)
- Troubleshooting: [docs/troubleshooting.md](docs/troubleshooting.md)
- Changelog: [CHANGELOG.md](CHANGELOG.md)
## Quickstart
Use either of these equivalent quickstart paths:
```bash
# Package without install
python -m pip install uv
uvx --from pyro-mcp pyro doctor
uvx --from pyro-mcp pyro env list
uvx --from pyro-mcp pyro env pull debian:12
uvx --from pyro-mcp pyro run debian:12 -- git --version
```
![Quickstart walkthrough](docs/assets/first-run.gif)
```bash
# Already installed
pyro doctor
pyro env list
pyro env pull debian:12
pyro run debian:12 -- git --version
```
From a repo checkout, replace `pyro` with `uv run pyro`.
What success looks like:
```bash
Platform: linux-x86_64
Runtime: PASS
Catalog version: 3.1.0
...
[pull] phase=install environment=debian:12
[pull] phase=ready environment=debian:12
Pulled: debian:12
...
[run] phase=create environment=debian:12
[run] phase=start vm_id=...
[run] phase=execute vm_id=...
[run] environment=debian:12 execution_mode=guest_vsock exit_code=0 duration_ms=...
git version ...
```
The first pull downloads an OCI environment from public Docker Hub, requires outbound HTTPS
access to `registry-1.docker.io`, and needs local cache space for the guest image.
## Stable Workspace Path
`pyro run` is the stable one-shot entrypoint. `pyro workspace ...` is the stable path when an
agent needs one sandbox to stay alive across repeated commands, shells, services, checkpoints,
diffs, exports, and reset.
The commands below use plain `pyro ...`. Run the same flow with `uvx --from pyro-mcp pyro ...`
for the published package, or `uv run pyro ...` from a source checkout.
```bash
uv tool install pyro-mcp
WORKSPACE_ID="$(pyro workspace create debian:12 --seed-path ./repo --json | python -c 'import json,sys; print(json.load(sys.stdin)["workspace_id"])')"
pyro workspace sync push "$WORKSPACE_ID" ./changes
pyro workspace exec "$WORKSPACE_ID" -- cat note.txt
pyro workspace snapshot create "$WORKSPACE_ID" checkpoint
pyro workspace service start "$WORKSPACE_ID" web --ready-file .web-ready -- sh -lc 'touch .web-ready && while true; do sleep 60; done'
pyro workspace reset "$WORKSPACE_ID" --snapshot checkpoint
pyro workspace export "$WORKSPACE_ID" note.txt --output ./note.txt
pyro workspace delete "$WORKSPACE_ID"
```
![Stable workspace walkthrough](docs/assets/workspace-first-run.gif)
That stable workspace path gives you:
- initial host-in seeding with `--seed-path`
- later host-in updates with `workspace sync push`
- one-shot commands with `workspace exec` and persistent PTYs with `workspace shell *`
- long-running processes with `workspace service *`
- explicit checkpoints with `workspace snapshot *`
- full-sandbox recovery with `workspace reset`
- baseline comparison with `workspace diff`
- explicit host-out export with `workspace export`
- secondary stopped-workspace disk inspection with `workspace stop|start` and `workspace disk *`
After the quickstart works:
- prove the full one-shot lifecycle with `uvx --from pyro-mcp pyro demo`
- create a persistent workspace with `uvx --from pyro-mcp pyro workspace create debian:12 --seed-path ./repo`
- update a live workspace from the host with `uvx --from pyro-mcp pyro workspace sync push WORKSPACE_ID ./changes`
- enable outbound guest networking for one workspace with `uvx --from pyro-mcp pyro workspace create debian:12 --network-policy egress`
- add literal or file-backed secrets with `uvx --from pyro-mcp pyro workspace create debian:12 --secret API_TOKEN=expected --secret-file PIP_TOKEN=./token.txt`
- map one persisted secret into one exec, shell, or service call with `--secret-env API_TOKEN`
- diff the live workspace against its create-time baseline with `uvx --from pyro-mcp pyro workspace diff WORKSPACE_ID`
- capture a checkpoint with `uvx --from pyro-mcp pyro workspace snapshot create WORKSPACE_ID checkpoint`
- reset a broken workspace with `uvx --from pyro-mcp pyro workspace reset WORKSPACE_ID --snapshot checkpoint`
- export a changed file or directory with `uvx --from pyro-mcp pyro workspace export WORKSPACE_ID note.txt --output ./note.txt`
- open a persistent interactive shell with `uvx --from pyro-mcp pyro workspace shell open WORKSPACE_ID`
- start long-running workspace services with `uvx --from pyro-mcp pyro workspace service start WORKSPACE_ID app --ready-file .ready -- sh -lc 'touch .ready && while true; do sleep 60; done'`
- publish one guest service port to the host with `uvx --from pyro-mcp pyro workspace create debian:12 --network-policy egress+published-ports` and `uvx --from pyro-mcp pyro workspace service start WORKSPACE_ID app --ready-http http://127.0.0.1:8080/ --publish 18080:8080 -- ./start-app`
- stop a workspace for offline inspection with `uvx --from pyro-mcp pyro workspace stop WORKSPACE_ID`
- inspect or export one stopped guest rootfs with `uvx --from pyro-mcp pyro workspace disk list WORKSPACE_ID`, `uvx --from pyro-mcp pyro workspace disk read WORKSPACE_ID note.txt`, and `uvx --from pyro-mcp pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4`
- move to Python or MCP via [docs/integrations.md](docs/integrations.md)
## Supported Hosts
Supported today:
- Linux x86_64
- Python 3.12+
- `uv`
- `/dev/kvm`
Optional for outbound guest networking:
- `ip`
- `nft` or `iptables`
- privilege to create TAP devices and configure NAT
Not supported today:
- macOS
- Windows
- Linux hosts without working KVM at `/dev/kvm`
## Detailed Walkthrough
If you want the expanded version of the canonical quickstart, use the step-by-step flow below.
### 1. Check the host
```bash
uvx --from pyro-mcp pyro doctor
```
Expected success signals:
```bash
Platform: linux-x86_64
Runtime: PASS
KVM: exists=yes readable=yes writable=yes
Environment cache: /home/you/.cache/pyro-mcp/environments
Capabilities: vm_boot=yes guest_exec=yes guest_network=yes
Networking: tun=yes ip_forward=yes
```
### 2. Inspect the catalog
```bash
uvx --from pyro-mcp pyro env list
```
Expected output:
```bash
Catalog version: 3.1.0
debian:12 [installed|not installed] Debian 12 environment with Git preinstalled for common agent workflows.
debian:12-base [installed|not installed] Minimal Debian 12 environment for shell and core Unix tooling.
debian:12-build [installed|not installed] Debian 12 environment with Git and common build tools preinstalled.
```
### 3. Pull the default environment
```bash
uvx --from pyro-mcp pyro env pull debian:12
```
The first pull downloads an OCI environment from public Docker Hub, requires outbound HTTPS
access to `registry-1.docker.io`, and needs local cache space for the guest image.
See [docs/host-requirements.md](docs/host-requirements.md) for the full host requirements.
### 4. Run one command in a guest
```bash
uvx --from pyro-mcp pyro run debian:12 -- git --version
```
Expected success signals:
```bash
[run] environment=debian:12 execution_mode=guest_vsock exit_code=0 duration_ms=...
git version ...
```
The guest command output and the `[run] ...` summary are written to different streams, so they
may appear in either order in terminals or capture tools. Use `--json` if you need a
deterministic structured result.
### 5. Optional demos
```bash
uvx --from pyro-mcp pyro demo
uvx --from pyro-mcp pyro demo --network
```
`pyro demo` proves the one-shot create/start/exec/delete VM lifecycle works end to end.
Example output:
```json
{
"cleanup": {
"deleted": true,
"reason": "post_exec_cleanup",
"vm_id": "..."
},
"command": "git --version",
"environment": "debian:12",
"execution_mode": "guest_vsock",
"exit_code": 0,
"stdout": "git version ...\n"
}
```
When you are done evaluating and want to remove stale cached environments, run `pyro env prune`.
If you prefer a fuller copy-pasteable transcript, see [docs/first-run.md](docs/first-run.md).
The walkthrough GIF above was rendered from [docs/assets/first-run.tape](docs/assets/first-run.tape) using [scripts/render_tape.sh](scripts/render_tape.sh).
## Stable Workspaces
Use `pyro run` for one-shot commands. Use `pyro workspace ...` when you need repeated commands in one
workspace without recreating the sandbox every time.
The project direction is an agent workspace, not a CI job runner. Persistent
workspaces are meant to let an agent stay inside one bounded sandbox across multiple
steps. See [docs/vision.md](docs/vision.md) for the product thesis and the
longer-term interaction model.
```bash
pyro workspace create debian:12 --seed-path ./repo
pyro workspace create debian:12 --network-policy egress
pyro workspace create debian:12 --seed-path ./repo --secret API_TOKEN=expected
pyro workspace create debian:12 --network-policy egress+published-ports
pyro workspace sync push WORKSPACE_ID ./changes --dest src
pyro workspace exec WORKSPACE_ID -- cat src/note.txt
pyro workspace exec WORKSPACE_ID --secret-env API_TOKEN -- sh -lc 'test "$API_TOKEN" = "expected"'
pyro workspace diff WORKSPACE_ID
pyro workspace snapshot create WORKSPACE_ID checkpoint
pyro workspace reset WORKSPACE_ID --snapshot checkpoint
pyro workspace reset WORKSPACE_ID
pyro workspace export WORKSPACE_ID src/note.txt --output ./note.txt
pyro workspace shell open WORKSPACE_ID --secret-env API_TOKEN
pyro workspace shell write WORKSPACE_ID SHELL_ID --input 'pwd'
pyro workspace shell read WORKSPACE_ID SHELL_ID
pyro workspace shell close WORKSPACE_ID SHELL_ID
pyro workspace service start WORKSPACE_ID web --secret-env API_TOKEN --ready-file .web-ready -- sh -lc 'touch .web-ready && while true; do sleep 60; done'
pyro workspace service start WORKSPACE_ID worker --ready-file .worker-ready -- sh -lc 'touch .worker-ready && while true; do sleep 60; done'
pyro workspace service start WORKSPACE_ID app --ready-http http://127.0.0.1:8080/ --publish 18080:8080 -- ./start-app
pyro workspace service list WORKSPACE_ID
pyro workspace service status WORKSPACE_ID web
pyro workspace service logs WORKSPACE_ID web --tail-lines 50
pyro workspace service stop WORKSPACE_ID web
pyro workspace service stop WORKSPACE_ID worker
pyro workspace stop WORKSPACE_ID
pyro workspace disk list WORKSPACE_ID
pyro workspace disk read WORKSPACE_ID src/note.txt
pyro workspace disk export WORKSPACE_ID --output ./workspace.ext4
pyro workspace start WORKSPACE_ID
pyro workspace logs WORKSPACE_ID
pyro workspace delete WORKSPACE_ID
```
Persistent workspaces start in `/workspace` and keep command history until you delete them. For
machine consumption, add `--json` and read the returned `workspace_id`. Use `--seed-path` when
you want the workspace to start from a host directory or a local `.tar` / `.tar.gz` / `.tgz`
archive instead of an empty workspace. Use `pyro workspace sync push` when you want to import
later host-side changes into a started workspace. Sync is non-atomic in `3.1.0`; if it fails
partway through, prefer `pyro workspace reset` to recover from `baseline` or one named snapshot.
Use `pyro workspace diff` to compare the live `/workspace` tree to its immutable create-time
baseline, and `pyro workspace export` to copy one changed file or directory back to the host. Use
`pyro workspace snapshot *` and `pyro workspace reset` when you want explicit checkpoints and
full-sandbox recovery. Use `pyro workspace exec` for one-shot
non-interactive commands inside a live workspace, and `pyro workspace shell *` when you need a
persistent PTY session that keeps interactive shell state between calls. Use
`pyro workspace service *` when the workspace needs one or more long-running background processes.
Typed readiness checks prefer `--ready-file`, `--ready-tcp`, or `--ready-http`; keep
`--ready-command` as the escape hatch. Service metadata and logs live outside `/workspace`, so the
internal service state does not appear in `pyro workspace diff` or `pyro workspace export`.
Use `--network-policy egress` when the workspace needs outbound guest networking, and
`--network-policy egress+published-ports` plus `workspace service start --publish` when one
service must be probed from the host on `127.0.0.1`.
Use `--secret` and `--secret-file` at workspace creation when the sandbox needs private tokens or
config. Persisted secrets are materialized inside the guest at `/run/pyro-secrets/<name>`, and
`--secret-env SECRET_NAME[=ENV_VAR]` maps one secret into one exec, shell, or service call without
exposing the raw value in workspace status, logs, diffs, or exports. Use `pyro workspace stop`
plus `pyro workspace disk list|read|export` when you need offline inspection or one raw ext4 copy
from a stopped guest-backed workspace, then `pyro workspace start` to resume the same workspace.
## Public Interfaces
The public user-facing interface is `pyro` and `Pyro`. After the CLI validation path works, you can choose one of three surfaces:
- `pyro` for direct CLI usage, including one-shot `run` and persistent `workspace` workflows
- `from pyro_mcp import Pyro` for Python orchestration
- `pyro mcp serve` for MCP clients
Command forms:
- published package without install: `uvx --from pyro-mcp pyro ...`
- installed package: `pyro ...`
- source checkout: `uv run pyro ...`
`Makefile` targets are contributor conveniences for this repository and are not the primary product UX.
## Official Environments
Current official environments in the shipped catalog:
- `debian:12`
- `debian:12-base`
- `debian:12-build`
The package ships the embedded Firecracker runtime and a package-controlled environment catalog.
Official environments are pulled as OCI artifacts from public Docker Hub repositories into a local
cache on first use or through `pyro env pull`.
End users do not need registry credentials to pull or run official environments.
The default cache location is `~/.cache/pyro-mcp/environments`; override it with
`PYRO_ENVIRONMENT_CACHE_DIR`.
## CLI
List available environments:
```bash
pyro env list
```
Prefetch one environment:
```bash
pyro env pull debian:12
```
Run one command in an ephemeral VM:
```bash
pyro run debian:12 -- git --version
```
Run with outbound internet enabled:
```bash
pyro run debian:12 --network -- \
'python3 -c "import urllib.request; print(urllib.request.urlopen(\"https://example.com\", timeout=10).status)"'
```
Show runtime and host diagnostics:
```bash
pyro doctor
pyro doctor --json
```
`pyro run` defaults to `1 vCPU / 1024 MiB`.
It fails closed when guest boot or guest exec is unavailable.
Use `--allow-host-compat` only if you explicitly want host execution.
Run the MCP server after the CLI path above works:
```bash
pyro mcp serve
```
Run the deterministic demo:
```bash
pyro demo
pyro demo --network
```
Run the Ollama demo:
```bash
ollama serve
ollama pull llama3.2:3b
pyro demo ollama
```
## Python SDK
```python
from pyro_mcp import Pyro
pyro = Pyro()
result = pyro.run_in_vm(
environment="debian:12",
command="git --version",
timeout_seconds=30,
network=False,
)
print(result["stdout"])
```
Lower-level lifecycle control remains available:
```python
from pyro_mcp import Pyro
pyro = Pyro()
created = pyro.create_vm(
environment="debian:12",
ttl_seconds=600,
network=True,
)
vm_id = created["vm_id"]
pyro.start_vm(vm_id)
result = pyro.exec_vm(vm_id, command="git --version", timeout_seconds=30)
print(result["stdout"])
```
`exec_vm()` is a one-command auto-cleaning call. After it returns, the VM is already deleted.
Environment management is also available through the SDK:
```python
from pyro_mcp import Pyro
pyro = Pyro()
print(pyro.list_environments())
print(pyro.inspect_environment("debian:12"))
```
For repeated commands in one workspace:
```python
from pyro_mcp import Pyro
pyro = Pyro()
workspace = pyro.create_workspace(environment="debian:12", seed_path="./repo")
workspace_id = workspace["workspace_id"]
try:
pyro.push_workspace_sync(workspace_id, "./changes", dest="src")
result = pyro.exec_workspace(workspace_id, command="cat src/note.txt")
print(result["stdout"], end="")
finally:
pyro.delete_workspace(workspace_id)
```
## MCP Tools
Primary agent-facing tool:
- `vm_run(environment, command, vcpu_count=1, mem_mib=1024, timeout_seconds=30, ttl_seconds=600, network=false, allow_host_compat=false)`
Advanced lifecycle tools:
- `vm_list_environments()`
- `vm_create(environment, vcpu_count=1, mem_mib=1024, ttl_seconds=600, network=false, allow_host_compat=false)`
- `vm_start(vm_id)`
- `vm_exec(vm_id, command, timeout_seconds=30)` auto-cleans the VM after that command
- `vm_stop(vm_id)`
- `vm_delete(vm_id)`
- `vm_status(vm_id)`
- `vm_network_info(vm_id)`
- `vm_reap_expired()`
Persistent workspace tools:
- `workspace_create(environment, vcpu_count=1, mem_mib=1024, ttl_seconds=600, network_policy="off", allow_host_compat=false, seed_path=null, secrets=null)`
- `workspace_sync_push(workspace_id, source_path, dest="/workspace")`
- `workspace_exec(workspace_id, command, timeout_seconds=30, secret_env=null)`
- `workspace_export(workspace_id, path, output_path)`
- `workspace_diff(workspace_id)`
- `snapshot_create(workspace_id, snapshot_name)`
- `snapshot_list(workspace_id)`
- `snapshot_delete(workspace_id, snapshot_name)`
- `workspace_reset(workspace_id, snapshot="baseline")`
- `service_start(workspace_id, service_name, command, cwd="/workspace", readiness=null, ready_timeout_seconds=30, ready_interval_ms=500, secret_env=null, published_ports=null)`
- `service_list(workspace_id)`
- `service_status(workspace_id, service_name)`
- `service_logs(workspace_id, service_name, tail_lines=200)`
- `service_stop(workspace_id, service_name)`
- `shell_open(workspace_id, cwd="/workspace", cols=120, rows=30, secret_env=null)`
- `shell_read(workspace_id, shell_id, cursor=0, max_chars=65536)`
- `shell_write(workspace_id, shell_id, input, append_newline=true)`
- `shell_signal(workspace_id, shell_id, signal_name="INT")`
- `shell_close(workspace_id, shell_id)`
- `workspace_status(workspace_id)`
- `workspace_logs(workspace_id)`
- `workspace_delete(workspace_id)`
## Integration Examples
- Python one-shot SDK example: [examples/python_run.py](examples/python_run.py)
- Python lifecycle example: [examples/python_lifecycle.py](examples/python_lifecycle.py)
- Python workspace example: [examples/python_workspace.py](examples/python_workspace.py)
- MCP client config example: [examples/mcp_client_config.md](examples/mcp_client_config.md)
- Claude Desktop MCP config: [examples/claude_desktop_mcp_config.json](examples/claude_desktop_mcp_config.json)
- Cursor MCP config: [examples/cursor_mcp_config.json](examples/cursor_mcp_config.json)
- OpenAI Responses API example: [examples/openai_responses_vm_run.py](examples/openai_responses_vm_run.py)
- LangChain wrapper example: [examples/langchain_vm_run.py](examples/langchain_vm_run.py)
- Agent-ready `vm_run` example: [examples/agent_vm_run.py](examples/agent_vm_run.py)
## Runtime
The package ships an embedded Linux x86_64 runtime payload with:
- Firecracker
- Jailer
- guest agent
- runtime manifest and diagnostics
No system Firecracker installation is required.
`pyro` installs curated environments into a local cache and reports their status through `pyro env inspect` and `pyro doctor`.
The public CLI is human-readable by default; add `--json` for structured output.
## Contributor Workflow
For work inside this repository:
```bash
make help
make setup
make check
make dist-check
```
Contributor runtime sources live under `runtime_sources/`. The packaged runtime bundle under
`src/pyro_mcp/runtime_bundle/` contains the embedded boot/runtime assets plus manifest metadata;
end-user environment installs pull OCI-published environments by default. Use
`PYRO_RUNTIME_BUNDLE_DIR=build/runtime_bundle` only when you are explicitly validating a locally
built contributor runtime bundle.
Official environment publication is performed locally against Docker Hub:
```bash
export DOCKERHUB_USERNAME='your-dockerhub-username'
export DOCKERHUB_TOKEN='your-dockerhub-token'
make runtime-materialize
make runtime-publish-official-environments-oci
```
`make runtime-publish-environment-oci` auto-exports the OCI layout for the selected
environment if it is missing.
The publisher accepts either `DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN` or
`OCI_REGISTRY_USERNAME` and `OCI_REGISTRY_PASSWORD`.
Docker Hub uploads are chunked by default for large rootfs layers; if you need to tune a slow
link, use `PYRO_OCI_UPLOAD_TIMEOUT_SECONDS`, `PYRO_OCI_UPLOAD_CHUNK_SIZE_BYTES`, and
`PYRO_OCI_REQUEST_TIMEOUT_SECONDS`.
For a local PyPI publish:
```bash
export TWINE_PASSWORD='pypi-...'
make pypi-publish
```
`make pypi-publish` defaults `TWINE_USERNAME` to `__token__`.
Set `PYPI_REPOSITORY_URL=https://test.pypi.org/legacy/` to publish to TestPyPI instead.