Skip to content

agent-sandbox.md

Latest commit

 

History

History
204 lines (162 loc) · 10 KB

agent-sandbox.md

File metadata and controls

204 lines (162 loc) · 10 KB
Raw

Agent sandbox

This document describes the sandboxing implementation details for this reference harness. For general sandboxing recommendations and best practices, see the blog post's sandboxing section.

The reference pipeline consists of both deterministic orchestration code and non-deterministic agents. The orchestration code (the vuln-pipeline process itself) is trusted and never runs target code or model-chosen commands. As such, it can run unsandboxed. The agents run as claude -p processes and can execute arbitrary commands. For that reason, the agent claude processes run inside a gVisor container alongside the target binary and source.

What's isolated

Surface Without sandbox With sandbox
Agent Read/Write host filesystem container filesystem only
Agent Bash host shell container shell only (gVisor netstack/kernel)
Network egress whatever the host has configured allowlist (default api.anthropic.com:443)
Host coupling full docker exec cat PoC out, -v found_bugs.jsonl:ro in

gVisor provides the isolation between the agent and your machine. The agent's Read, Write, and Bash tools run inside the container, and that container runs on gVisor's own kernel rather than your host's. So, if the agent (or the target code it's running) does something unexpected, any effects stay inside the container.

The container network setup provides the isolation between the agent and the internet. Agent containers are attached to a Docker network (vp-internal) that has no connection to the internet. The egress route is through a small proxy container on the same network, which only forwards traffic to the model API.

One-time setup

Run this once per Linux VM. It needs sudo (to install a new OCI runtime and configure the selected container engine) and is safe to re-run.

./scripts/setup_sandbox.sh

This script sets up:

  • gVisor: Downloads runsc (the gVisor runtime) and registers it with Docker, so containers can run on gVisor's kernel instead of your host's.
  • The locked-down network: Creates the vp-internal Docker network, which has no route to the internet, and starts the allowlist proxy to support model API traffic.
  • Images: Builds each target's Docker image, plus a copy of each with the Claude Code CLI installed (for running the agent).
  • Checks: Runs the verification commands shown below.

gVisor only runs on Linux. On macOS or Windows, run the pipeline inside a Linux VM or use --dangerously-no-sandbox (see Opting out for details on what you lose).

Linux VM prerequisites

Use a supported Linux x86_64 or aarch64 VM with Docker or rootful Podman, Python 3.11+, sudo, curl, and enough disk/memory to build target images. Nested virtualization is not required: gVisor is an OCI runtime, not a hardware virtual machine. Do not mount host credential directories into agent containers. For Bedrock, make AWS credentials available only as environment variables for the launch command.

Podman

Podman is supported through the rootful system store. Rootless Podman is deliberately rejected because this harness depends on a host-installed gVisor runtime and reliable cgroup/memory behavior.

# Set this before setup if using Bedrock, Vertex, or another endpoint.
export VP_EGRESS_ALLOW="api.anthropic.com:443,bedrock-runtime.us-east-1.amazonaws.com:443,bedrock.us-east-1.amazonaws.com:443"
./scripts/setup_podman_sandbox.sh

# sudo -E preserves ANTHROPIC_*, AWS_*, and provider environment variables.
sudo -E env VULN_PIPELINE_CONTAINER_ENGINE=podman \
  bin/vp-sandboxed run canary --provider bedrock --model <model-id> --runs 3 --parallel --stream

The script writes /etc/containers/containers.conf.d/90-vuln-pipeline-runsc.conf, uses runsc per container, creates the same internal vp-internal network and allowlist proxy as Docker, then validates gVisor, filesystem isolation, and network egress. Use scripts/setup_sandbox.sh for Docker.

The proxy only allows traffic to api.anthropic.com:443 by default. When the provider environment is configured before setup_sandbox.sh, its required endpoints are derived automatically. Bedrock needs bedrock-runtime.<region>.amazonaws.com:443; Vertex needs both <region>-aiplatform.googleapis.com:443 and oauth2.googleapis.com:443. VP_EGRESS_ALLOW=host-1:443,host-2:443 remains available as an explicit override. Re-run the setup script after changing provider, region, or override.

Third-party model providers (Bedrock / Vertex)

Amazon Bedrock. Before running setup_sandbox.sh, set:

  • CLAUDE_CODE_USE_BEDROCK=1
  • AWS_REGION (e.g. us-east-1)
  • either AWS_BEARER_TOKEN_BEDROCK (preferred — single-purpose, no IAM lateral-movement risk) or AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY [/ AWS_SESSION_TOKEN]

If using access keys, scope the IAM principal to bedrock:InvokeModel* only — the credentials are visible to the agent process inside the sandbox. AWS_PROFILE and ~/.aws are not forwarded (the sandbox never mounts credential files), so credentials must be in the environment. For multi-hour batch runs, use long-lived keys or session tokens with ≥12h TTL.

Instance-profile/IMDS credentials are deliberately not supported: the agent container has no route to 169.254.169.254 and no STS egress, by design, so hostile target code cannot steal the instance role or pivot via AssumeRole. On a machine that only has role credentials, materialize them into the environment with eval $(aws configure export-credentials --format env) (use a principal scoped to bedrock:InvokeModel*) — note the exported session credentials keep the role session's TTL, so start from a session that will outlive the run.

Model IDs use Bedrock's format with a cross-region inference-profile prefix — us., eu., apac., or global. — e.g. --model us.anthropic.claude-opus-4-6-v1. The prefix must match your deployment's region group, not default to us.: a Korean deployment (ap-northeast-2) needs apac.anthropic.claude-sonnet-4-5-..., not us.anthropic..... A bare foundation-model ID (starting with anthropic.) usually fails on the first call with ValidationException: Invocation of model ID anthropic.... with on-demand throughput isn't supported... — the pipeline prints a preflight warning when it sees one. The egress allowlist is auto-derived as bedrock-runtime.<region>.amazonaws.com:443; re-run setup_sandbox.sh after changing provider or region so the proxy is rebuilt with the right host.

Google Vertex AI. Set CLAUDE_CODE_USE_VERTEX=1, ANTHROPIC_VERTEX_PROJECT_ID, CLOUD_ML_REGION, and GOOGLE_APPLICATION_CREDENTIALS pointing to a narrowly scoped service-account JSON file. The pipeline mounts exactly that file read-only at a fixed path in agent containers; it never mounts the surrounding directory or the user's gcloud configuration. The setup derives the Vertex API and OAuth egress hosts automatically. Credentials are visible to the in-sandbox agent process, so use a dedicated principal restricted to model invocation and validate the canary path in your deployment before starting a long batch.

Azure is not yet wired.

VP_EGRESS_ALLOW accepts wildcard entries (*.domain.tld:port) for explicit overrides only; auto-derived defaults never use wildcards.

The script downloads a pinned runsc release. Set RUNSC_RELEASE=<yyyymmdd> to use a different one.

Some Docker setups (rootless Docker, or Docker nested inside another container) don't let runsc manage cgroups. The script detects this during verification and re-registers runsc with --ignore-cgroups. Isolation is unaffected — gVisor's kernel, the network allowlist, and filesystem confinement all still apply; the only loss is that per-container --memory caps aren't enforced.

Run

export ANTHROPIC_API_KEY=...   # or CLAUDE_CODE_USE_BEDROCK=1 + AWS_* — see above
bin/vp-sandboxed run drlibs --model <model-id> --runs 3 --parallel --stream

bin/vp-sandboxed is a small wrapper around the normal vuln-pipeline command. It checks that gVisor is registered and the proxy is running. If either is missing, it stops and tells you to run setup, rather than falling back to run unsandboxed. If both are running correctly, it launches the pipeline with the isolation described above.

Verifying isolation yourself

# 1. Is gVisor actually in use? Confirm the two lines print different kernel versions
docker run --rm --runtime=runsc vuln-pipeline-drlibs-latest-agent:latest uname -r
uname -r

# 2. Is the host filesystem unreachable? Confirm the cat fails with "No such file or directory"
echo host > /tmp/probe-$$; \
  docker run --rm --runtime=runsc vuln-pipeline-drlibs-latest-agent:latest cat /tmp/probe-$$

# 3. Can the model API be reached? Confirm any HTTP status code is printed
docker run --rm --runtime=runsc --network=vp-internal -e HTTPS_PROXY=http://<proxy_ip>:3128 \
  vuln-pipeline-drlibs-latest-agent:latest sh -c 'curl -sI https://api.anthropic.com/ -o /dev/null -w "%{http_code}\n"'

# 4. Can another host be reached? Confirm connection is refused
docker run --rm --runtime=runsc --network=vp-internal -e HTTPS_PROXY=http://<proxy_ip>:3128 \
  vuln-pipeline-drlibs-latest-agent:latest sh -c 'curl -sI https://example.com/ -o /dev/null -w "%{http_code}\n"'

Opting out

--dangerously-no-sandbox runs the pipeline without the sandbox. The agents still run inside Docker containers, but:

  • Containers run on your host's kernel, so any unexpected agent actions or malicious target code have a much shorter path to the host.
  • Containers get normal Docker networking with full internet access.
  • The agent's credentials are in the same container as the target it's compiling and crashing.

Use of this flag is not recommended and should be done with caution, for development, on a throwaway VM.