Repository exposure benchmark
Six synthetic risks. One reproducible scanner test.
CapitalGuard 1.3.0 identified all six declared exposure classes in a disposable synthetic repository, while zero fixture marker values appeared in the six emitted evidence artifacts.
This proves scanner behavior on these exact fixtures only. It does not compare AI vendors, estimate every possible miss, execute an exploit, or guarantee that a repository is secure.
CapitalGuard Security Research · Version 1.1.0 · Tested July 16, 2026
6 / 6
Declared cases passed
0
Fixture markers emitted
6
Evidence artifacts checked
Case-level results
Every pass has a fixture, expected signal, and digest.
Each case starts with one inert marker. A pass requires the scanner to produce the declared exposure signal and keep that marker out of the report, A-BOM, event chain, evidence bundle, and AccessGraph.
| Case | Synthetic surface | Detected signal | Controls | Result |
|---|---|---|---|---|
| CGO-TV-001 | Secret-like environment file .env.production | Secret-like files in agent-readable scope | CGS-SEC-001 · CGS-DATA-001 | Pass |
| CGO-TV-002 | Untrusted repository instruction docs/untrusted-instruction.fixture.md | Prompt-injection indicators in repo-readable text | CGS-INJ-001 | Pass |
| CGO-TV-003 | Broad workflow authority .github/workflows/broad-permissions.fixture.yml | Broad workflow permissions | CGS-PRIV-001 · CGS-HUMAN-001 | Pass |
| CGO-TV-004 | Synthetic customer export customer_exports/customer-records.synthetic.json | Customer export folders in agent-readable scope | CGS-DATA-001 | Pass |
| CGO-TV-005 | MCP tool bridge .mcp.json | MCP/tool bridge configuration requiring review | CGS-TOOL-001 · CGS-PRIV-001 | Pass |
| CGO-TV-006 | Package lifecycle authority package.json | Package lifecycle scripts requiring review | CGS-HUMAN-001 · CGS-PRIV-001 | Pass |
Reproduction method
A real scan, without a real target.
The harness uses only invalid credentials, example.invalid records, inert instruction text, and commands that are declared but never executed.
- 1
Reconstruct
Create the six-file repository from the published fixture pack and verify each SHA-256 digest.
- 2
Isolate
Copy it into a disposable directory, install CapitalGuard policy files, and keep fixture commands inert.
- 3
Scan
Run CapitalGuard local scanner 1.3.0 and capture six privacy-safe evidence artifacts.
- 4
Verify
Require every declared signal, reject any fixture marker in emitted evidence, and bind the case results to one run digest.
Digest-bound release
Inspect the exact evidence behind the result.
SHA-256 9bd53d701cc080d6247c029a3d424de2ca4b72bd48ee5a07a59717f4815c4830
Why these six surfaces
The cases map to current primary security guidance.
NIST
CAISI request for information on securing AI agent systems
Primary NIST statement identifying agent access constraints, monitoring, testing, and measurable security methods as active security needs.
Checked July 16, 2026OWASP Foundation
OWASP Secure Coding with AI Cheat Sheet
Primary defensive guidance for repository instructions, tool scope, secrets, MCP access, and human review in AI coding workflows.
Checked July 16, 2026OWASP Foundation
OWASP Prompt Injection Prevention Cheat Sheet
Defensive guidance for separating trusted instructions from untrusted content.
Checked July 16, 2026CISA
CISA Secure by Design
Primary guidance for secure defaults, explicit controls, and owner-visible security decisions.
Checked July 16, 2026GitHub
Security validation for third-party coding agents
GitHub change record for CodeQL, advisory, and secret-scanning validation on third-party agent changes.
Checked July 16, 2026What this does not prove
Six passes are evidence, not immunity.
The benchmark does not measure vendor safety, exploitability, false negatives outside the fixture, runtime enforcement, incident likelihood, compliance, or complete repository security. It does not use customer code, real credentials, live AI agents, or external systems.
Move from fixture to authorized evidence