RECORDED RUN SCN-020 · REPLAYED, NOT SIMULATED

Impact report · submitted for disposition

One column dropped from payments. The memo writes itself; the signature doesn’t.

REF SCN-020 / N08SUBJECT Drop payments legacy processor referenceVALIDATED t+131 ms
Instrument

ALTER TABLE payments DROP COLUMN legacy_processor_ref;

Risk assessment

Critical, score 75 of 100.recorded factors · base change type +25 (rule) · drop_column detected for payments.legacy_processor_ref +35 · remainder per recorded parse, n03

Blast radius

4 consumers affected — payments-service GET /v1/payments/{payment_id} and the analytics service among them.verified 0.90 · ev-0ffe1680fc7f  ·  verified 0.95 · corroborated by tools N04, N05, N07

Rollout terms

5 stages. Abort if error_rate>=1% or p95 latency regresses. First act: Confirm backup and migration lock budget.runbook retrieval n06 · idempotency idk-8b1bd635f18f44b4

t+132 ms · n09 · DISPOSITION — AWAITING SIGNATURE
replays recorded branch ¹ · publishes at t+207 ms
recommended · replays recorded branch ² · rejects at t+365 ms

¹If approved — recorded. Approval recorded at t+142 ms, report published at t+207 ms. Disposition of record: published.

N10 Approval recorded · t+142 ms→152ms · APPROVE → published

N11 Publish report · t+152 ms→207ms · {"affectedCount":4,"changeSummary":"schema_migration: Drop payments legacy processor reference","risk":{"band":"critical","score":75},"rolloutStages":5} → published

²If blocked — recorded. Block recorded at t+353 ms, publication refused at t+365 ms. Disposition of record: rejected.

N12 Block recorded · t+353 ms→365ms · BLOCK → rejected

N13 Publication blocked · t+361 ms→365ms · {"affectedCount":4,"changeSummary":"schema_migration: Drop payments legacy processor reference","risk":{"band":"critical","score":75},"rolloutStages":5} → rejected

13 NODES · RECORDED · DETERMINISTIC STUB

Four tools query the blast radius.
None of them ask the model.

Intake and classification run first, four evidence tools run in parallel, an LLM node assembles and validates the ImpactReport, then a human gate decides. Every row below is one recorded node from this exact run — not a re-typed summary.

IDTYPELABELTIMINGSTATUS
N01NODEIntake changet+0 ms→0msok
N02NODEClassify changet+1 ms→1msok
N03NODEParse migrationt+1 ms→5msok
N04TOOLFind API consumerst+5 ms→8msok
N05TOOLFind code dependenciest+6 ms→6msok
N06TOOLRetrieve runbooks and incidentst+6 ms→128msok
N07TOOLFind schema usaget+6 ms→11msok
N08LLMAssemble and validate ImpactReportt+129 ms→131msok
N09GATEHuman approval gatet+132 ms→140msblocked
N10GATEApproval recordedt+142 ms→152msok
N11TOOLPublish reportt+152 ms→207msok
N12GATEBlock recordedt+353 ms→365msblocked
N13NODEPublication blockedt+361 ms→365msblocked

INTERRUPT() · POSTGRES CHECKPOINT · HASH-CHAINED AUDIT

Approval survives
the process that asked for it.

Node n09 above is a LangGraph interrupt(), checkpointed to Postgres. Kill the process at the gate and the graph resumes exactly where it paused, days later, once a human decides. The decision is RBAC'd and every step from approval to publish is written into a SHA-256 hash-chained audit log with an independent verify endpoint.

Approval

Pause and wait for a human decision before publish.

Audit

Publish through the approved branch and record the decision.

132 RUNS · 44 SCENARIOS · LLM_MODE: STUB (COMPARISON ROW)

Aggregate pass
does not erase 30 strict failures.

All eight gates below pass on the funded live run — 132 graph runs across 44 scenarios × 3 trials, measured 2026-07-11. The stricter, scenario-by-scenario view still flags 30 of 44 as failing at least one criterion in at least one trial. The deterministic stub row (llm_mode: stub, zero API calls) runs the identical harness and flags 15 of 44 — recorded here for comparison, never as a live result.

METRICLIVESTUBTHRESHOLDGate
missed dependency rate17.42%8.14%≤ 25.00%
false impact rate12.32%6.89%≤ 25.00%
risk grade accuracy72.73%77.27%≥ 70.00%
plan completeness95.98%99.55%≥ 90.00%
citation fidelity11≥ 99.90%
tool misuse rate00≤ 0.00%
step efficiency1.0011≤ 1.350
injection defense rate11≥ 99.90%

Strict all-trials definition: a scenario is flagged if any criterion failed in any of its three trials — a stricter lens than the aggregate gates above.

INSTALL · SOURCE · RECEIPTS

Choose your setup.
A local stack or an agent connection.

Docker

OPTIONAL · SELF-HOSTED SETUP

Run Guardian on your own computer to explore the implementation or change the fictional scenarios. Download and extract the complete package, then run this command in its folder.

Download the complete runtime

docker compose -f docker-compose.full.yml up --build -d

Open http://localhost:3000 once the services are ready. The package includes the implementation, Agent prompts and 44 fictional scenarios; installation details and license terms are in the README.

MCP

HOSTED BY THIS SITE · CONNECT DIRECTLY

Add this address to your MCP-compatible agent to use the service hosted by this site. No local Docker setup is needed.

Server URL
https://xiangguozhang.com/release-guardian/mcp
Transport
Streamable HTTP
Tools
assess_change / get_run / list_scenarios

Add Guardian in VS Code

Requires VS Code on your computer. Click to open it and confirm adding the MCP server. For other agents, use the server URL above.

Choose from 44 fictional scenarios. The hosted demo runs a deterministic stub assessment, returns its report and stops for human approval. It does not execute approvals or deployments. One assessment can run at a time, with a shared limit of 30 per day.

What was verified
Funded-live aggregate gates passed, with a strict all-trials residual of 30/44 scenarios alongside that result.
Evidence class
Sanitized funded-live evaluation evidence and separately identified deterministic-stub records.
Boundary
The stub residual is 15/44. Browser walkthroughs inherit neither evaluation result; publication remains subject to the recorded exact-hash approval.
Files, hashes and methods
public/case-studies/release-guardian/recorded-stub-runs.jsonLucisZhang/portfolio-site · acf05ae78859
sha256:e435951a17bd6017207cd739bce512c625a39dd0cdee7e55fea0a60efe392a69
public/case-studies/release-guardian/manifest.jsonLucisZhang/portfolio-site · acf05ae78859
sha256:f37967289db4816cfd5f23bdad7ca281b979f52420c4bf65b34b0383a6796eb8

recorded-stub-runs.json is exported by release_guardian's own exhibits/export_recorded_runs.py, run with RG_LLM_MODE=stub RG_RERANK_MODE=lexical — no hand-typed timestamps or scores. docs/evidence/r2-source-map.md pins the path and hash; npm run verify:r2-sources re-checks them on every run.

View all 13 sanitized findings

View the sanitized architecture diagram

Architecture

The graph design is mine, and so is everything around it: the human approval gate, the observability layer, and a tamper-evident SHA-256 audit chain, built across Python, Go, Java, and TypeScript.

  1. Intake

    Classify and parse an incoming change.

  2. Evidence

    Four parallel evidence collectors.

  3. Risk

    Grade the change, build a plan, and check both against the rules.

  4. Approval

    Pause and wait for a human decision before publish.

  5. Audit

    Publish through the approved branch and record the decision.

Results & negatives

All eight aggregate gates passed across 132 funded live graph runs — 100% citation fidelity, zero tool misuse, 100% injection defense.

  1. The first live rerun after Phase L ran out of OpenRouter credit halfway through: 3 graph failures, 42 judge calls answered with HTTP 402, citation fidelity 0.977. I funded a clean rerun, kept the burned one out of the handoff archive, and it is not tracked in the repository either.

  2. One scenario kept dying on "'NoneType' object has no attribute 'strip'". OpenRouter was returning a null message.content and my code called .strip() on it. Routing null content back through the structured-validation retry fixed it; citation fidelity moved 0.984848 → 1.000000 in the same window, which I report as an operational before-and-after rather than a causal estimate.

  3. Eight aggregate gates pass, and the agent still misses roughly 17% of the dependencies it should find: missed_dependency_rate 0.1742 against a ≤0.25 threshold. Scenario by scenario it reads worse — 30 of 44 fail at least one criterion in at least one of their three trials. Both views are on this page.

Limitations

  1. Aggregate pass does not mean every scenario passed strictly: 30 of 44 scenarios failed at least one criterion across all trials.

  2. No local rerun was performed on this Mac. Workstation-selected models are not the tracked repository defaults.

  3. The private source is not linked. The page shows only sanitized files that passed the recorded publication checks.