Inspiration

AI agents now refund orders, run shell commands, edit databases and deploy code, and most of them do it with no seatbelt. When an agent does something wrong, it usually does it at machine speed. Knight Capital, the GitLab database deletion and the Hawaii false missile alert were all cases of a system acting before a human could say "wait." Teams that ship agents today get either blind trust or a log they read after the damage is done.

We wanted something like a flight recorder and seatbelt for AI agents. It records everything an agent does and pauses the dangerous things until a human says yes.

What it does

Tripwire sits between an agent and the actions it wants to take.

  1. The agent reports every action to Tripwire before running it.
  2. Gemini scores the action's risk. Built-in rules set a floor the model can't talk down.
  3. Low-risk actions run immediately. Risky ones pause until a human approves them with step-up MFA. Denied actions never run.
  4. Everything is logged and charted live: volume, a risk trace and an event stream.

Beyond the core loop:

  • Investigator chat: ask Gemini "what did billing-bot do between 2 and 3pm?"
  • Incident summaries and similar-incident search powered by Gemini.
  • Explainable scores: every action has a "Why this score" breakdown. It shows whether the score came from a built-in rule, Gemini, unusual behavior for that agent, past human decisions, or a policy.
  • Your own policies: write rules that block an action, always ask a human, or allow it. Loosening a rule needs a fresh MFA check.
  • Fail-closed by default: if Tripwire is unreachable, the agent is blocked. A timed-out request never lets an action run.
  • Drop-in adapters: JavaScript and Python SDKs, an MCP proxy that guards any stdio MCP server with no code changes, and a Claude Code PreToolUse hook.

How we built it

  • Next.js 16 / React 19 / Tailwind for the web app and the ingest API.
  • Auth0 does three jobs. Humans sign in. Agents get machine-to-machine tokens. Approving a risky action requires step-up MFA.
  • Tiger Data (TimescaleDB) stores agent events in a hypertable. Continuous aggregates power the live dashboard.
  • Gemini handles risk scoring, the Investigator chat, incident summaries and similar-incident search.
  • Vercel hosts the app, with a custom domain through GoDaddy DNS.
  • Entire recorded our Claude Code sessions as checkpoints while we built, so the project's development is fully auditable.
  • A small agent SDK with a scripted simulator drives a repeatable 3-minute demo: normal work, a risky action gets blocked, a human approves with MFA, then the Investigator explains what happened.

The scoring pipeline has five layers:

$$\text{risk} = \text{clamp}\big(\max(r_{\text{rules}},\, r_{\text{Gemini}}) + \Delta_{\text{baseline}} + \Delta_{\text{feedback}}\big)$$

Policies run first. A deny rule blocks with no model call. The built-in rules and Gemini give the base score, and the higher of the two wins. A per-agent behavioral baseline then adds a bump for a never-seen action type or an activity spike. Past human decisions on similar actions adjust the score up or down.

Challenges we ran into

  • Making failure safe. The hard question isn't "what if the model is wrong?" but "what if Tripwire is down?" We built explicit fail modes (closed, allowlist, open) and only allow the permissive ones when the request provably never reached the server. After a timeout or a 5xx the action may already be waiting for a human, so it stays blocked in every mode.
  • Agents write their own text. An allow-policy that matched free text could be gamed by the agent itself, so allow rules can only match a whole action type or target. We also reject regex patterns that could cause catastrophic backtracking.
  • Not letting the model talk itself down. Gemini can lower a score, but never below the built-in rules. Critical rules, require_approval policies and holds caused by a Gemini outage are never relaxed by feedback.
  • A demo that works every time. Live LLM demos fail at the worst moment. We added RISK_MODEL=off, which scores with rules and policies only. That removes the dependency on Gemini quota and makes every run identical.
  • Wiring five sponsor tools into one coherent product, each for a real purpose and not just a logo on a slide.

Accomplishments that we're proud of

  • A working end-to-end loop where a real agent action is held, approved with real MFA and logged.
  • A read-only viewer mode, so judges can explore the console and ask the Investigator without being able to approve anything.
  • An MCP proxy that lets Tripwire guard existing tools with zero code changes.

What we learned

  • Governance for agents is mostly a trust-boundary problem. The agent's own words can't be trusted as an input to its own permissions.
  • Time-series storage fits agent telemetry well. Hypertables and continuous aggregates made a live dashboard cheap to build.
  • Step-up MFA is the right primitive for human-in-the-loop approval, because it proves a person, not just a session, said yes.
  • Explainability matters as much as accuracy. People approve faster and more safely when they can see why an action was flagged.

What's next for Tripwire

  • Slack and mobile push approvals
  • Per-team approval routing and quorum approvals for the riskiest actions
  • More adapters (OpenAI Agents SDK, LangGraph) and a hosted multi-tenant version
  • Anomaly detection trained on each organization's own approval history

Built With

Share this project:

Updates

Submission history