pyro-mcp/README.md
Thales Maciel aa886b346e Add seeded task workspace creation
Current persistent tasks started with an empty workspace, which blocked the first useful host-to-task workflow in the task roadmap. This change lets task creation start from a host directory or tar archive without changing the one-shot VM surfaces.

Expose source_path on task create across the CLI, SDK, and MCP, add safe archive upload and extraction support for guest and host-compat backends, persist workspace_seed metadata, and patch the per-task rootfs with the bundled guest agent before boot so seeded guest tasks work without republishing environments. Also switch post--- command reconstruction to shlex.join() so documented sh -lc task examples preserve argument boundaries.

Validation:
- uv lock
- UV_CACHE_DIR=.uv-cache uv run pytest --no-cov tests/test_vm_guest.py tests/test_vm_manager.py tests/test_cli.py tests/test_api.py tests/test_server.py tests/test_public_contract.py
- UV_CACHE_DIR=.uv-cache make check
- UV_CACHE_DIR=.uv-cache make dist-check
- real guest-backed smoke: task create --source-path, task exec -- cat note.txt, task delete
2026-03-11 21:45:38 -03:00

456 lines
13 KiB
Markdown

# pyro-mcp
`pyro-mcp` runs one-shot commands and repeated task workspaces inside ephemeral Firecracker microVMs using curated Linux environments such as `debian:12`.
[![PyPI version](https://img.shields.io/pypi/v/pyro-mcp.svg)](https://pypi.org/project/pyro-mcp/)
This is for coding agents, MCP clients, and developers who want isolated command execution in ephemeral microVMs.
It exposes the same runtime in three public forms:
- the `pyro` CLI
- the Python SDK via `from pyro_mcp import Pyro`
- an MCP server so LLM clients can call VM tools directly
## Start Here
- Install: [docs/install.md](docs/install.md)
- First run transcript: [docs/first-run.md](docs/first-run.md)
- Terminal walkthrough GIF: [docs/assets/first-run.gif](docs/assets/first-run.gif)
- PyPI package: [pypi.org/project/pyro-mcp](https://pypi.org/project/pyro-mcp/)
- What's new in 2.2.0: [CHANGELOG.md#220](CHANGELOG.md#220)
- Host requirements: [docs/host-requirements.md](docs/host-requirements.md)
- Integration targets: [docs/integrations.md](docs/integrations.md)
- Public contract: [docs/public-contract.md](docs/public-contract.md)
- Troubleshooting: [docs/troubleshooting.md](docs/troubleshooting.md)
- Changelog: [CHANGELOG.md](CHANGELOG.md)
## Quickstart
Use either of these equivalent quickstart paths:
```bash
# Package without install
python -m pip install uv
uvx --from pyro-mcp pyro doctor
uvx --from pyro-mcp pyro env list
uvx --from pyro-mcp pyro env pull debian:12
uvx --from pyro-mcp pyro run debian:12 -- git --version
```
![Quickstart walkthrough](docs/assets/first-run.gif)
```bash
# Already installed
pyro doctor
pyro env list
pyro env pull debian:12
pyro run debian:12 -- git --version
```
From a repo checkout, replace `pyro` with `uv run pyro`.
What success looks like:
```bash
Platform: linux-x86_64
Runtime: PASS
Catalog version: 2.2.0
...
[pull] phase=install environment=debian:12
[pull] phase=ready environment=debian:12
Pulled: debian:12
...
[run] phase=create environment=debian:12
[run] phase=start vm_id=...
[run] phase=execute vm_id=...
[run] environment=debian:12 execution_mode=guest_vsock exit_code=0 duration_ms=...
git version ...
```
The first pull downloads an OCI environment from public Docker Hub, requires outbound HTTPS
access to `registry-1.docker.io`, and needs local cache space for the guest image.
After the quickstart works:
- prove the full one-shot lifecycle with `uvx --from pyro-mcp pyro demo`
- create a persistent workspace with `uvx --from pyro-mcp pyro task create debian:12 --source-path ./repo`
- move to Python or MCP via [docs/integrations.md](docs/integrations.md)
## Supported Hosts
Supported today:
- Linux x86_64
- Python 3.12+
- `uv`
- `/dev/kvm`
Optional for outbound guest networking:
- `ip`
- `nft` or `iptables`
- privilege to create TAP devices and configure NAT
Not supported today:
- macOS
- Windows
- Linux hosts without working KVM at `/dev/kvm`
## Detailed Walkthrough
If you want the expanded version of the canonical quickstart, use the step-by-step flow below.
### 1. Check the host
```bash
uvx --from pyro-mcp pyro doctor
```
Expected success signals:
```bash
Platform: linux-x86_64
Runtime: PASS
KVM: exists=yes readable=yes writable=yes
Environment cache: /home/you/.cache/pyro-mcp/environments
Capabilities: vm_boot=yes guest_exec=yes guest_network=yes
Networking: tun=yes ip_forward=yes
```
### 2. Inspect the catalog
```bash
uvx --from pyro-mcp pyro env list
```
Expected output:
```bash
Catalog version: 2.2.0
debian:12 [installed|not installed] Debian 12 environment with Git preinstalled for common agent workflows.
debian:12-base [installed|not installed] Minimal Debian 12 environment for shell and core Unix tooling.
debian:12-build [installed|not installed] Debian 12 environment with Git and common build tools preinstalled.
```
### 3. Pull the default environment
```bash
uvx --from pyro-mcp pyro env pull debian:12
```
The first pull downloads an OCI environment from public Docker Hub, requires outbound HTTPS
access to `registry-1.docker.io`, and needs local cache space for the guest image.
See [docs/host-requirements.md](docs/host-requirements.md) for the full host requirements.
### 4. Run one command in a guest
```bash
uvx --from pyro-mcp pyro run debian:12 -- git --version
```
Expected success signals:
```bash
[run] environment=debian:12 execution_mode=guest_vsock exit_code=0 duration_ms=...
git version ...
```
The guest command output and the `[run] ...` summary are written to different streams, so they
may appear in either order in terminals or capture tools. Use `--json` if you need a
deterministic structured result.
### 5. Optional demos
```bash
uvx --from pyro-mcp pyro demo
uvx --from pyro-mcp pyro demo --network
```
`pyro demo` proves the one-shot create/start/exec/delete VM lifecycle works end to end.
Example output:
```json
{
"cleanup": {
"deleted": true,
"reason": "post_exec_cleanup",
"vm_id": "..."
},
"command": "git --version",
"environment": "debian:12",
"execution_mode": "guest_vsock",
"exit_code": 0,
"stdout": "git version ...\n"
}
```
When you are done evaluating and want to remove stale cached environments, run `pyro env prune`.
If you prefer a fuller copy-pasteable transcript, see [docs/first-run.md](docs/first-run.md).
The walkthrough GIF above was rendered from [docs/assets/first-run.tape](docs/assets/first-run.tape) using [scripts/render_tape.sh](scripts/render_tape.sh).
## Persistent Tasks
Use `pyro run` for one-shot commands. Use `pyro task ...` when you need repeated commands in one
workspace without recreating the sandbox every time.
```bash
pyro task create debian:12 --source-path ./repo
pyro task exec TASK_ID -- sh -lc 'printf "hello from task\n" > note.txt'
pyro task exec TASK_ID -- cat note.txt
pyro task logs TASK_ID
pyro task delete TASK_ID
```
Task workspaces start in `/workspace` and keep command history until you delete them. For machine
consumption, add `--json` and read the returned `task_id`. Use `--source-path` when you want the
task to start from a host directory or a local `.tar` / `.tar.gz` / `.tgz` archive instead of an
empty workspace.
## Public Interfaces
The public user-facing interface is `pyro` and `Pyro`. After the CLI validation path works, you can choose one of three surfaces:
- `pyro` for direct CLI usage, including one-shot `run` and persistent `task` workflows
- `from pyro_mcp import Pyro` for Python orchestration
- `pyro mcp serve` for MCP clients
Command forms:
- published package without install: `uvx --from pyro-mcp pyro ...`
- installed package: `pyro ...`
- source checkout: `uv run pyro ...`
`Makefile` targets are contributor conveniences for this repository and are not the primary product UX.
## Official Environments
Current official environments in the shipped catalog:
- `debian:12`
- `debian:12-base`
- `debian:12-build`
The package ships the embedded Firecracker runtime and a package-controlled environment catalog.
Official environments are pulled as OCI artifacts from public Docker Hub repositories into a local
cache on first use or through `pyro env pull`.
End users do not need registry credentials to pull or run official environments.
The default cache location is `~/.cache/pyro-mcp/environments`; override it with
`PYRO_ENVIRONMENT_CACHE_DIR`.
## CLI
List available environments:
```bash
pyro env list
```
Prefetch one environment:
```bash
pyro env pull debian:12
```
Run one command in an ephemeral VM:
```bash
pyro run debian:12 -- git --version
```
Run with outbound internet enabled:
```bash
pyro run debian:12 --network -- \
'python3 -c "import urllib.request; print(urllib.request.urlopen(\"https://example.com\", timeout=10).status)"'
```
Show runtime and host diagnostics:
```bash
pyro doctor
pyro doctor --json
```
`pyro run` defaults to `1 vCPU / 1024 MiB`.
It fails closed when guest boot or guest exec is unavailable.
Use `--allow-host-compat` only if you explicitly want host execution.
Run the MCP server after the CLI path above works:
```bash
pyro mcp serve
```
Run the deterministic demo:
```bash
pyro demo
pyro demo --network
```
Run the Ollama demo:
```bash
ollama serve
ollama pull llama3.2:3b
pyro demo ollama
```
## Python SDK
```python
from pyro_mcp import Pyro
pyro = Pyro()
result = pyro.run_in_vm(
environment="debian:12",
command="git --version",
timeout_seconds=30,
network=False,
)
print(result["stdout"])
```
Lower-level lifecycle control remains available:
```python
from pyro_mcp import Pyro
pyro = Pyro()
created = pyro.create_vm(
environment="debian:12",
ttl_seconds=600,
network=True,
)
vm_id = created["vm_id"]
pyro.start_vm(vm_id)
result = pyro.exec_vm(vm_id, command="git --version", timeout_seconds=30)
print(result["stdout"])
```
`exec_vm()` is a one-command auto-cleaning call. After it returns, the VM is already deleted.
Environment management is also available through the SDK:
```python
from pyro_mcp import Pyro
pyro = Pyro()
print(pyro.list_environments())
print(pyro.inspect_environment("debian:12"))
```
For repeated commands in one workspace:
```python
from pyro_mcp import Pyro
pyro = Pyro()
task = pyro.create_task(environment="debian:12", source_path="./repo")
task_id = task["task_id"]
try:
pyro.exec_task(task_id, command="printf 'hello from task\\n' > note.txt")
result = pyro.exec_task(task_id, command="cat note.txt")
print(result["stdout"], end="")
finally:
pyro.delete_task(task_id)
```
## MCP Tools
Primary agent-facing tool:
- `vm_run(environment, command, vcpu_count=1, mem_mib=1024, timeout_seconds=30, ttl_seconds=600, network=false, allow_host_compat=false)`
Advanced lifecycle tools:
- `vm_list_environments()`
- `vm_create(environment, vcpu_count=1, mem_mib=1024, ttl_seconds=600, network=false, allow_host_compat=false)`
- `vm_start(vm_id)`
- `vm_exec(vm_id, command, timeout_seconds=30)` auto-cleans the VM after that command
- `vm_stop(vm_id)`
- `vm_delete(vm_id)`
- `vm_status(vm_id)`
- `vm_network_info(vm_id)`
- `vm_reap_expired()`
Persistent workspace tools:
- `task_create(environment, vcpu_count=1, mem_mib=1024, ttl_seconds=600, network=false, allow_host_compat=false, source_path=null)`
- `task_exec(task_id, command, timeout_seconds=30)`
- `task_status(task_id)`
- `task_logs(task_id)`
- `task_delete(task_id)`
## Integration Examples
- Python one-shot SDK example: [examples/python_run.py](examples/python_run.py)
- Python lifecycle example: [examples/python_lifecycle.py](examples/python_lifecycle.py)
- Python task workspace example: [examples/python_task.py](examples/python_task.py)
- MCP client config example: [examples/mcp_client_config.md](examples/mcp_client_config.md)
- Claude Desktop MCP config: [examples/claude_desktop_mcp_config.json](examples/claude_desktop_mcp_config.json)
- Cursor MCP config: [examples/cursor_mcp_config.json](examples/cursor_mcp_config.json)
- OpenAI Responses API example: [examples/openai_responses_vm_run.py](examples/openai_responses_vm_run.py)
- LangChain wrapper example: [examples/langchain_vm_run.py](examples/langchain_vm_run.py)
- Agent-ready `vm_run` example: [examples/agent_vm_run.py](examples/agent_vm_run.py)
## Runtime
The package ships an embedded Linux x86_64 runtime payload with:
- Firecracker
- Jailer
- guest agent
- runtime manifest and diagnostics
No system Firecracker installation is required.
`pyro` installs curated environments into a local cache and reports their status through `pyro env inspect` and `pyro doctor`.
The public CLI is human-readable by default; add `--json` for structured output.
## Contributor Workflow
For work inside this repository:
```bash
make help
make setup
make check
make dist-check
```
Contributor runtime sources live under `runtime_sources/`. The packaged runtime bundle under
`src/pyro_mcp/runtime_bundle/` contains the embedded boot/runtime assets plus manifest metadata;
end-user environment installs pull OCI-published environments by default. Use
`PYRO_RUNTIME_BUNDLE_DIR=build/runtime_bundle` only when you are explicitly validating a locally
built contributor runtime bundle.
Official environment publication is performed locally against Docker Hub:
```bash
export DOCKERHUB_USERNAME='your-dockerhub-username'
export DOCKERHUB_TOKEN='your-dockerhub-token'
make runtime-materialize
make runtime-publish-official-environments-oci
```
`make runtime-publish-environment-oci` auto-exports the OCI layout for the selected
environment if it is missing.
The publisher accepts either `DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN` or
`OCI_REGISTRY_USERNAME` and `OCI_REGISTRY_PASSWORD`.
Docker Hub uploads are chunked by default for large rootfs layers; if you need to tune a slow
link, use `PYRO_OCI_UPLOAD_TIMEOUT_SECONDS`, `PYRO_OCI_UPLOAD_CHUNK_SIZE_BYTES`, and
`PYRO_OCI_REQUEST_TIMEOUT_SECONDS`.
For a local PyPI publish:
```bash
export TWINE_PASSWORD='pypi-...'
make pypi-publish
```
`make pypi-publish` defaults `TWINE_USERNAME` to `__token__`.
Set `PYPI_REPOSITORY_URL=https://test.pypi.org/legacy/` to publish to TestPyPI instead.