Inspiration
During a production incident, the "right" fix depends on who you ask. The reliability engineer wants to restart the database, security wants to preserve evidence of an intrusion, and the customer team wants to know whether payments will be lost. Real postmortems (Knight Capital, GitLab, Cloudflare) are full of moments where the familiar fix made things worse or the wrong people weren't in the room. We were inspired by the MAGI supercomputer from Neon Genesis Evangelion: three minds with different priorities that must agree before acting. We wanted to see whether AI agents with genuinely competing objectives could do the same, and know when to stop and ask a human.
What it does
MAGI convenes three agents, MELCHIOR (reliability), BALTHASAR (security and integrity) and CASPER (customers and obligations), when an alert fires. Each sees only the evidence its role is entitled to. They propose plans blindly, score and rank them, critique and amend, register mandatory dissent, and vote on an exact, hashed version of a plan. There is no chair: a code-only facilitator enforces the rules.
- Reversible actions pass with 2 of 3 votes and run automatically through n8n.
- Irreversible actions need all 3 votes plus a human tap in Slack.
- Deadlocks trigger a phone call from an ElevenLabs voice agent that briefs the on-call engineer on the crux, can unseal sensitive evidence with a code, and records their ruling.
- Every change of mind must name its cause (new evidence, a new argument, its own error, or "peer agreement only", which is flagged as possible conformity).
- Everything is event-sourced: incidents survive restarts and produce a decision record.
How we built it
- Backend: Python, FastAPI and asyncio. The council is a state machine with Pareto filtering, Schulze ranked voting, risk tiers computed from live system state, and per-mandate veto domains. Agents run on the OpenAI Responses API with native tool calls, each limited to its own data tools.
- Frontend: React, Vite, Tailwind and Zustand. An Evangelion-inspired control room that is a pure function of the event log.
- n8n Cloud: alert intake, the agents' tool gateway, signed action execution, Slack approvals, the escalation call, and the voice-tool relay.
- ElevenLabs + Twilio: the outbound phone agent with four server tools, plus a signature-verified post-call webhook.
- Render with a persistent disk for hosting and incident history.
- Evaluation harness: 13 decision points (6 adapted from public postmortems, 4 synthetic and our 2 demos), each labeled in advance with best, acceptable and harmful actions. Each runs under three conditions: one all-seeing agent, three agents that just vote, and the full council.
Challenges we ran into
- Making disagreement real, not scripted: we had to design private evidence and written policies so the agents' objectives genuinely conflict.
- Plumbing: tunnels that expired mid-test, n8n dynamic webhook paths, Twilio trial restrictions, and an error message n8n quietly swallowed (MAGI said "n8n unavailable" when Twilio was actually refusing the call).
- Reasoning models used their whole output budget thinking, so we had to raise the token floors.
- Our own evaluation humbled us. It caught scoring bugs in our labels and, more importantly, real protocol bugs. One rule escalated for "insufficient information" even when every agent had proposed the information-gathering step itself. And the decision time limit was too short for a slow reasoning model.
What we learned
- Disagreement is a feature, but only if the system can tell productive dissent from a stall.
- In our first live run, no condition recommended a harmful action and all three caught the planted false evidence. However, the simple vote escalated more accurately (77%) than the full debate (31%). That matches research suggesting debate adds little over voting on raw accuracy. Its value is in auditability and catching specific traps, and the cost is roughly 30x the tokens of a single agent.
- Measure early. Without the harness we would have shipped a council that pages people too often and believed it was better.
What's next
Fix the two protocol issues the eval found, rerun with multiple runs and independent labelers, add agents on different model families, and connect the tool gateway to real observability data.
Log in or sign up for Devpost to join the conversation.