Covenant

Inspiration

For the past few years most of my income has come from building automated trading systems. Bots that place real orders with real money while I'm asleep. If you've ever run one of those for more than a month, you know the feeling: the system will eventually do something you never wrote a rule against, and it'll do it confidently at 3am. So everything I ship in that world has hard risk caps, dry-run modes and boring audit logs. Not because it's elegant. Because I got burned enough times to stop trusting suggestions.

Then I started watching AI agents get the exact same powers my trading bots have. Sending emails, publishing things, exporting customer data. And the only thing standing between the agent and a disaster is... a paragraph in a system prompt. That's not a control, that's a wish.

The part that really bothered me is what happens after a mistake. You correct the agent in chat, it apologizes, and that correction just evaporates into the scrollback. Three weeks later a model update brings the same mistake back. In trading, every incident I've ever hit became a permanent test case that runs before any new version goes live. I couldn't find anything that did this for agents, so I built it during Build Week.

What it does

Covenant sits between an agent and its tools. You write your rules in plain English, things like "never send customer data to an external domain" or "anything contacting more than 5 people needs my approval". GPT-5.6 compiles that into a typed policy, which you review before it does anything.

From then on, every tool call the agent proposes gets checked by a plain TypeScript engine before it executes. Allow, block, or ask a human. There is no model call anywhere in that decision, and if the engine hits an error it fails closed. Every proposal, decision, approval and execution gets appended to a hash-linked receipt you can verify in the browser and export. Flip one byte in the export and verification fails.

The piece I care most about is the correction loop. When something goes wrong, you tell Covenant what should have happened, in a sentence. GPT-5.6 turns that into a regression test plus a proposed policy change. Then a deterministic replay proves the new test fails under the old policy and passes under the new one, with all 12 existing tests still green, before you decide whether to activate it. The agent can't edit its own guardrails. In the demo, the agent tries to send a customer-data report to an external partner and the engine blocks it before the tool runs. There's an execution-recorder test proving that path literally cannot fire.

How we built it

All the core code was written by Codex in a single project thread, milestone by milestone against a spec I locked upfront. I made the scope and architecture calls, reviewed each gate, verified independently and committed. The models have three narrow jobs: policy compilation, the agent's own tool proposals, and correction compilation, all through the Responses API with strict structured outputs and Zod validation on top. gpt-5.6-terra handles cheap failure classification. The full build log with real test output is in EVIDENCE.md in the repo.

Hardest part honestly was discipline, not code. The whole design is about keeping the LLM out of the trust boundary, and every shortcut tempts you to let the model just decide. Proving a negative was also fun, "the blocked tool never ran" needed a proper test harness, not a screenshot. And running in-memory sandbox state on serverless means a cold start can eat your session, so the UI just says so and offers a fresh run instead of pretending it can't happen.

51 tests pass, plus an end-to-end browser test that runs green against the live production URL. Built solo, from an empty folder, inside the submission window.

Challenges we ran into

The hardest part honestly was discipline, not code. The whole design is about keeping the LLM out of the trust boundary, and every shortcut tempts you to just let the model decide. Proving a negative was also interesting: "the blocked tool never ran" needed a proper execution-recorder test harness, not a screenshot. And running in-memory sandbox state on serverless means a cold start can eat your session, so the UI just tells you that and offers a fresh run instead of pretending it can't happen. Smaller fights along the way: strict JSON schemas rejecting model output until the retry-with-diagnostics path was right, and a first classifier call that came back empty and had to be retried.

Accomplishments that we're proud of

51 tests pass, plus an end-to-end browser test that runs green against the live production URL, not just localhost. A live six-turn GPT-5.6 run produces allow, allow, allow, require-approval, block, exactly as designed. One typed sentence of correction becomes a policy v2 with 13/13 tests green and zero regressions. And there's a small SDK you can prove in one command: npx tsx examples/minimal-agent.ts. Solo build, empty folder to deployed product, inside the submission window.

What we learned

Governance is a product, not a prompt. Once enforcement became deterministic and corrections became permanent tests, the agent stopped being something I had to watch and became something I could audit. That's the same transition that made me trust my trading systems years ago. Also: the model is genuinely great at compiling human intent into structure, and genuinely the wrong thing to put in the enforcement path. Use it where it's strong, fence it where it isn't.

What's next for Covenant

An MCP adapter so any MCP tool can sit behind the engine, starter policy packs for common agent roles, and real signatures on receipts (right now they're honestly labeled as tamper-evident within the export, nothing more). The EU AI Act's record-keeping rules start applying August 2, 2026. Agents are going to need receipts.

Try it, no login or key needed: https://covenant-umber.vercel.app — launch the golden task, approve the bulk send, watch the leak get blocked, then approve the patch and watch 12/13 turn into 13/13.

Built With

• codex
• gpt-5.6
• next.js
• node.js
• openai
• playwright
• react
• responses-api
• tailwind
• typescript
• vercel
• vitest
• zod