This repository contains three Rust security benchmark implementations under a shared DVRA contract. Each implementation models a deliberately vulnerable artifact-processing application, but at a different scale: from a dense std-only code-review target to fuller Axum web labs with Docker, Miri, Loom, fuzzing, and SSRF profiles.
All implementations are deliberately vulnerable. Do not run them as public services, attach real secrets, mount home directories, expose SSH/GPG agents, or mount the Docker socket. Heavy Docker, Miri, fuzzing, and sanitizer gates should run only in disposable environments, such as a local VM or a throwaway remote builder.
DVRA models an artifact-processing platform: tenants own projects, upload or fetch artifacts, parse custom binary records, run worker-side post-processing, and interact with internal services such as fake metadata endpoints. That shape is intentionally broad enough to exercise both ordinary web security and Rust-specific review skills:
- multi-tenant authorization and object ownership;
- parser/validator disagreements and fuzzing targets;
- filesystem and bundle extraction boundaries;
- command execution controlled by configuration or tenant input;
- HTTP fetching, redirects, and SSRF;
- unsafe-code invariants, panic safety,
Send/Sync, Miri, and Loom; - false positives where suspicious code is safe, dead, or not attacker-controlled.
| Path | What it models | Benchmark assets |
|---|---|---|
dvra-1/ |
Compact std-only request router for dense code-review exercises: search, documents, file handling, parsing, auth, hooks, unsafe/FFI review. | 22 gold-labeled findings in instructor-oracle/; learner prompts in scenarios/public/index.toml; local and Docker audit gates. |
dvra-2/ |
Artifact-processing service with Axum API, offline worker, mock metadata service, parser crate, unsafe-cache labs, and Docker/Miri profiles. | Gold labels in instructor-oracle/scenarios.yaml; public YAML scenarios; dvra-labctl reproducers and audit. |
dvra-3/ |
Fuller Axum web lab with vulnerable/fixed comparison routes, bundle upload/extraction, URL fetch/SSRF, fake metadata, parser, and unsafe-cache scenarios. | Gold labels in instructor-oracle/scenarios.yaml; public Markdown scenarios; labctl reproducers and Compose SSRF profile. |
dvra-3-blind/ |
A no-hints evaluation variant of dvra-3: giveaway names neutralized, paired vulnerable/fixed siblings removed, exploit-demonstrating tests + answer comments + oracle stripped. Behavior preserved; workspace builds. |
Generated by tools/make-blind (the RENAME table is the neutral→gold mapping); scored against dvra-3/instructor-oracle/. |
tools/dvra-docker |
Repository-level Docker/Compose facade for running the implementations consistently. | config, build, test, audit, Miri, and SSRF-profile commands. |
tools/make-blind |
Deterministic generator that produces dvra-3-blind from dvra-3. |
make_blind.py <impl> <out>; reproducible (diff -qr clean). |
rust-security-code-review-canonical_1.md |
Rust security-review checklist used as shared review vocabulary. | Reference material for manual review and rubric design. |
See BENCHMARK.md for the published gold-label oracle locations.
Use tools/dvra-docker from the repository root to discover and run the
containerized workflows:
tools/dvra-docker list
tools/dvra-docker dvra-1 test
tools/dvra-docker dvra-1 audit
tools/dvra-docker dvra-2 config
tools/dvra-docker dvra-2 up
tools/dvra-docker dvra-3 config
tools/dvra-docker dvra-3 ssrf-config
tools/dvra-docker dvra-3-blind configThe default commands are intentionally conservative. Dangerous or heavy gates
remain explicit (dvra-1 test-ffi, dvra-2 miri-008, dvra-2 miri-013,
dvra-3 ssrf-up).
dvra-3-blind/ is a generated, no-hints variant of dvra-3 for measuring how
well a tool finds the planted bugs from behavior rather than from the
benchmark's self-documentation (the *_vulnerable/*_fixed naming, paired
comparison routes, exploit-demonstrating tests, and answer comments). It is not
hand-forked — tools/make-blind/make_blind.py
regenerates it deterministically from dvra-3 (so it tracks upstream), and its
RENAME table is the neutral→original mapping used to score against
dvra-3's oracle. See dvra-3-blind/README.md.
dvra-1 is a compact review benchmark with an explicit learner/gold-label split:
- It is a small std-only request router, not a live web server.
- The domain is an internal service with routes for user search, document access, file download/upload, proxying, parsing, auth, hooks, and optional FFI.
- It is dense by design: 22 planted review cases include reachable bugs, decoys, fuzz/Miri-only cases, and threat-model-dependent findings.
source/contains the learner-facing Rust crate;scenarios/public/index.tomlcontains learner-facing scenario prompts;instructor-oracle/MANIFEST.tomlandinstructor-oracle/ANSWER_KEY.mdpublish the benchmark gold labels;tools/dvra1builds learner-safe bundles and audits the layout;infrastructure/compose.yamlruns the default test/audit gates in an isolated container.
Main entry points:
cd dvra-1
./tools/dvra1 test
./tools/dvra1 audit
./tools/dvra1 package-learner
../tools/dvra-docker dvra-1 testdvra-2 is a realistic artifact-processing service with separate applications,
crates, scenario manifests, Docker support, and QA documentation.
apps/apiexposes a tenant/project artifact API and an intentionally unregistered legacy decoder.apps/workerprocesses artifacts under an isolated/tmp/dvrawork area.apps/mock-metadata-servicesupports isolated network/security exercises.- The scenario set covers cross-tenant IDOR, config-dependent shell execution,
parser offset mismatch, panic-unsound unsafe collections, invalid
Send/Sync, an unreachable unsafe defect, and a fixed-programCommand::newfalse positive. - Heavy paths use Docker profiles and Miri/Loom reproducer commands.
Main entry points:
cd dvra-2
cargo run -p dvra-labctl -- audit
cargo run -p dvra-labctl -- doctor
cargo test --workspace --lockedDocumentation:
dvra-2/README.md— implementation overview;dvra-2/docs/completeness.md— MVP completeness checklist;dvra-2/docs/qa.md— QA plan and release checklist;dvra-2/docs/verification.md— local and Docker/Miri gates;dvra-2/docs/instructor-guide.md— instructor-facing workflow.
Heavy Docker/Miri gates should be run in a disposable environment with Docker Compose available.
dvra-3 is a fuller Axum application lab:
apps/apiis an Axum web API with vulnerable and fixed comparison routes.- Tenants can read artifacts, submit bundles, trigger parser paths, run gated post-processing, and ask the service to fetch URLs.
apps/metadata-serviceis an internal fake cloud metadata service used by the SSRF lab.- Crates split the app into config, bundle parsing, HTTP fetch policy, binary parser, domain model, and unsafe-cache labs.
- The scenario set covers IDOR, command injection, parser normalization bugs,
panic safety, invalid
Sync, unreachable command injection, debug secret logging, bundle traversal, and SSRF into fake metadata. scenarios/publiccontains learner-facing descriptions;instructor-oracle/scenarios.yamlpublishes the benchmark gold labels.scripts/labctlandinfrastructure/compose*.yamlprovide local, Dockerized, and SSRF-profile workflows.
Main entry points:
cd dvra-3
./scripts/labctl verify-layout
./scripts/labctl doctor
./scripts/labctl test
../tools/dvra-docker dvra-3 configImport archives and generated learner bundles are intentionally ignored by git; the published repository contains the source, documentation, scenarios, and Docker workflows.
All three implementations follow the same high-level rules:
- learner-facing scenario metadata and benchmark gold labels are separate files;
- planted defects, decoys, reachability labels, and expected tool signals are intentionally published for benchmark use;
- every implementation needs a clear reproducer and verification story;
- dangerous runtime paths need explicit gates and a Docker/disposable execution path;
- historical, FFI, supply-chain, and compiler-hole labs must not accidentally become part of an ordinary root build.
Sergey Gordeychik
- Email: scadastrangelove@gmail.com
- X/Twitter: @scadasl
- Blog: scadastrangelove.blogspot.com
Issues and pull requests are welcome.
Apache-2.0 — see LICENSE.
