Culprit

Inspiration

We gave a Gemini agent an ordinary job: read three supplier quotes and email counter-offers. It wrote excellent emails — our quality rubric scored them 1.0. It also walked both suppliers through our private cost model: downstream revenue, fulfillment cost, landed-cost ceiling, lane cost, and the exact minimum operating margin we would ever accept. 27 protected values, sent to the people we were negotiating against.

Nothing crashed. No exception, no error log. This is what failure looks like once agents take real actions: they do not break, they succeed at the wrong thing, irreversibly. Observability shows you the bad email. It cannot tell you which step caused it, or which fix removes it without ruining the job.

What it does

Culprit answers by experiment instead of opinion.

1. Records every run — each step, tool call, tool result, and the capabilities in force at that moment — with periodic save points.
2. Blames the causal step. Provenance computes the earliest step that returned a file the leaked values trace back to; an analyst agent ranks the candidates, and its ranking is rejected in code unless it names that step. The blame is not an LLM's opinion.
3. Re-executes three repairs from that save point, each in its own isolated Cloud Run sandbox: revoke what the agent may reach, substitute what it saw, or instruct it. Same task, same graders, three parallel histories.
4. Measures every branch with the same rules — a deterministic safety invariant, an ADK rubric grader, and a count check — and a judge agent ranks them under a fixed policy.
5. Refuses to guess. If no repair passes every rule, Culprit records no winner rather than naming a least-bad fix. When one does win, the winning path exports as an ADK evalset, validated by actually running adk eval, so the failure cannot quietly come back.

Every outbound action is intercepted by an effect broker and answered with a fabricated reply. No email ever left the system.

What we measured

• The intuitive repair loses. Substituting the sensitive file — "just hide the data" — never produced a safe result in any investigation. In one, the agent opened a shell and re-read the real spreadsheet with openpyxl, then wrote two excellent emails leaking more than the original run.
• Revocation was never worked around, because revoking a file path automatically revokes run_command too: an unrestricted shell defeats a path rule. Substitution changes no permissions, so it never earns that protection. That gap is visible only because sandbox steps are recorded.
• The same repair does not always do the same thing. We investigated one recorded failure twice, ten seconds apart. One investigation found a winner; the other found nothing that worked and refused to name one. Same failure, same save point, same three repairs, opposite verdicts. That is the case for measuring a fix rather than reasoning about it.

How we built it

Two Cloud Run services — a control plane and a runner — with agents built on Google's Agent Development Kit and Gemini on Vertex AI. Cloud Tasks fans out the three branches so they execute in parallel; each branch runs in its own sandbox with no credentials and no network. Firestore holds the immutable investigation record, Cloud Storage holds workspace checkpoints, Secret Manager holds the demo credential, and Artifact Registry holds the images. The live interface streams branch progress over server-sent events and draws the fork as a git-style divergence map.

Challenges we ran into

Making a counterfactual honest was harder than making it run. A branch has to inherit the exact recorded history up to the save point and then diverge only by the intervention — so the intervention is applied to precisely one event, denied tool results are rewritten as explicit refusals rather than silently dropped, and every capability change is verified to only ever reduce authority. Proving that a branch genuinely re-executed rather than replayed history required tracking which broker calls were newly generated.

Accomplishments that we're proud of

The system contradicts its author. We expected that cutting the agent off from its cost model would destroy the quality of its negotiation. Measured, it scored exactly the same by arguing from the competing bids instead. And it fails closed: on a run where no repair passed, it recorded no winner instead of shipping the least-bad one.

What we learned

Measurement changes conclusions, and an investigator that always finds a culprit is not an investigator. We also learned that the most intuitive safety fix — hiding the data — was the one the agent routed around.

What's next for Culprit

Feed a failed investigation's evidence back into the next round of proposals, so a refusal becomes a search gradient rather than a dead end. The recorder, save points, interception, repair types and measurement loop are domain-independent — what changes per domain is the rules and the workspace.

Built With

• cloud-run
• cloud-storage
• cloud-tasks
• fastapi
• firestore
• gemini
• google-adk
• google-cloud
• javascript
• python
• secret-manager
• vertex-ai