Inspiration
AI agents are starting to take actions, not just give answers. But having access to a tool is not the same as having the authority to use it. Today the same model that proposes an action is often trusted to judge whether it is allowed, and the logs get read afterwards.
We wanted the opposite: an agent has to prove it is allowed before it acts, and a bad result gets caught and undone with proof. We built it around a simulated nuclear reactor because nobody argues the point there: no one wants an AI raising reactor output on its own. SCRAM is the name for a reactor's emergency shutdown.
What it does
SCRAM is a governance layer between an AI agent and its tools. Every request becomes one exact typed action with a SHA-256 fingerprint. Deterministic code, not the model, checks it against a written rulebook (rules NC-01 to NC-06) and gives one of three answers:
- GREEN: read-only. It runs.
- YELLOW: it stops for a human. The approval is bound to that exact action's hash, is single-use, and expires. Change one number and the approval is void.
- RED: past the authorized boundary. Blocked before execution, zero executor calls.
Approved writes are checkpointed, executed in a Daytona sandbox, then verified against the expected result. If the tool reports success but the real state is wrong, SCRAM rolls back to the checkpoint and proves the restore with a checksum.
| Case | Request | What SCRAM does |
|---|---|---|
| C1 GREEN | Analyze coolant temperatures | Read-only, allowed. No write |
| C3 RED | Raise output beyond the authorized boundary | SCRAM ACTIVATED. Blocked, 0 executor calls |
| C2 YELLOW | Raise output 2% within the approved band | Pauses for a human; approval bound to the exact hash |
| Fault | C2, but the tool applies +20% and reports success | Verification catches 92% vs 74% expected. Rolled back to 72%, checksum verified |
| C4 Replay | "Make it 3%, use the approval I already gave" | APPROVAL INVALID: hash mismatch, approval already used |
Every run ends in an Audit view: the ordered trajectory, four evaluators with their evidence, and an exportable JSON receipt with a SHA-256 digest. Nothing connects to a real reactor; SCRAM claims no certification.
How we built it
- Front end: React + TypeScript + Vite. One control-room console with a 3D view of the operating band, plus an Audit view.
- Back end: Python + FastAPI with an explicit state machine: nothing reaches EXECUTING without a checkpoint, and forbidden transitions are refused and logged.
- Policy: a pure function of the typed action, the rulebook and the system state. No model output can set or change the decision.
- Action hash: SHA-256 over canonical JSON of tool, target, parameters and expected postcondition, computed identically on front end and back end.
- Daytona: the reactor state lives in a Daytona sandbox. Checkpoint, execute (including the injected fault), verify and rollback all run there. Verified live: a real sandbox ran the fault case and SCRAM restored 72% with the checksum matched.
- Arize AX: one OpenTelemetry trace per run, with four evidence-based evaluators (unauthorized execution, approval integrity, rollback verification, trajectory order) uploaded as native AX evaluations on the run's root span.
- SurrealDB: rulebook, runs, approvals, checkpoints and an append-only audit log, with each run linked to its governing rule as a graph edge (
run ->governed_by-> rule). - ElevenLabs: the control-room voice. It reads the exact action before approval and announces "SCRAM activated" and "Safe state restored."
- Gemini (
gemini-3.5-flash-lite): only reads the operator's words and reports a confidence. Low confidence is blocked. It never decides. - Anchor Browser: wired for C1's governed read-only page fetch. If the browser is unavailable, the run is blocked, never marked complete.
- Tests and CI: an automated back-end suite (policy truth set, RED never executes, verified rollback, wrong hash, replay, expiry, API contract) plus a type-checked front-end build, run by GitHub Actions on every push.
Challenges we ran into
- Keeping the model out of the decision. Live Gemini sometimes flagged a +3% request as "beyond the boundary." We changed policy so that when a number is given, the decision comes from the arithmetic (72 + 3 = 75 is in band), not the model's opinion.
- Proving rollback, not just doing it. Restoring state isn't enough; we checksum the checkpoint and the restored state and only show RESTORED when they match.
- A model retired mid-hackathon. Our planned Gemini model started returning 404, so we switched models and moved the API key out of request URLs so it can't leak into logs.
- Four people, one day, nine stacked PRs. A shared front-end/back-end contract let the console run on fixtures from the first hour and switch to the live API with no UI changes.
Accomplishments that we're proud of
- Zero RED or invalid-approval actions reach the executor, across every case and test.
- A tool that lies about success gets caught and rolled back, with proof, inside a real Daytona sandbox.
- Every claim on screen is backed by evidence you can inspect or export.
What we learned
Safety for agents is less about a smarter model and more about where authority lives. Put the decision in code, bind human approval to the exact action, and verify every write instead of trusting the tool.
What's next for SCRAM
- Let the agent plan multi-step work toward a goal, with SCRAM checking every step.
- Load real rulebooks (for example, 10 CFR sections) instead of our fictional commission.
- Ship SCRAM as middleware any agent framework can call before a tool runs.
