Inspiration

Every job had a backup. The plan still broke.

I haven't run a food distribution myself, but I've watched that sentence play out more than once: a college event, a family function, a group trip. Everyone had a job, someone was "handling" the backup, and then two small things went wrong on the same day and nobody had a plan for that combination.

What bothered me was why those plans looked finished. Every task had a name next to it, so everyone relaxed. But a backup only helps if it doesn't depend on the same thing as the person it's covering. And the failures that hurt are rarely one big thing. They're two small things at once, each harmless alone. Nobody checks those combinations by hand, because there are too many of them.

I chose a Saturday food distribution as the first case, because when that kind of plan breaks, the people who pay for it are families waiting for food.

What it does

Failure Compiler is a crash test for a plan.

You paste the plan the way coordinators actually write it, like a group-chat message. AI turns it into plain facts, such as "Maya leaves at 10:45", each linked to the line of the plan it came from, and openly marked when the plan doesn't say. You confirm or correct every fact. Then a strict checker, not the AI, tries every single problem and every pair of problems from a fixed list: someone doesn't show up, someone leaves early, a vehicle is back late, the supply truck is late, the loading area is blocked.

For the example (30 households, 3 routes, 5 volunteers, a van, a car, one loading area, everything delivered by noon) that's 100 tests, and they run in about a second and a half:

100 tests: 1 single weak spot, 3 hidden pairs, 12 already broken by the weak spot, 84 survived.

The weak spot is the obvious one: Joel has the only key. The hidden pairs are the point. The van coming back 30 minutes late is fine. Anita leaving 90 minutes early is fine. On the same morning, the van gets back at 10:45, everyone who could load it has gone, and Route C's 8 households get no food.

Then you ask the AI for fixes, and every suggestion is scored against all 100 tests before you can accept it. In the example, the AI suggested 12 fixes and the checker turned down 10. For the van and Anita case it turned down all three, each for a different reason:

"Add Ravi as an additional driver for Route C" was turned down because Ravi was already a driver. The route was never short of a driver. It was short of a loader.
"Extend Anita's availability" actually set her finish time earlier. The description and the change disagreed, and only the checker noticed.
"Assign the car to Route C" fixed three crosses but made "supply truck late" a brand-new single weak spot.

The AI's suggestion to give one volunteer a spare key was turned down too, because it created a new hidden pair. The one that worked was to let Sam, who drives the van back, load it as well, and both van crosses flipped to ticks.

The AI's fixes get checked, and so do yours. For the last two crosses, I built my own changes from the same allowed options (spare keys for two volunteers, and Sam as a backup driver for Route A). They went through the same gate and the same 100 tests, and every scenario survived.

A Before/After switch shows the original grid next to the fixed one, and a printable report opens with what broke, what fixed it and what was never tested.

The rule behind all of it: AI reads and suggests. The checker decides.

There's no hosted version to sign up for: the repo's judge guide runs the full example locally in one server, with no key, account or Docker needed.

How I built it

Plan first. I built this with the Devpost Learn Skill Pack, running all six skills in order. The scope, PRD and spec were each approved and committed before any code existed, and they're all in the repo. The planning changed the product more than the code did:

Scope cut my "verify any plan in any industry" idea down to one sharp case.
PRD caught that one weak spot would fill a whole row of the grid with crosses and bury the hidden pairs, so those cells are now dimmed.
Spec forced me to separate what the AI returns from what the server trusts, and to define exactly who may fill each job. That's what makes "add a backup driver" something the checker can test, and turn down.

Checker first, screens later. The first build slice had no interface and no AI: just the checker, plus tests that lock in the example's story. Every slice after that had to pass its tests before it was committed. The project now has 238 automated tests, including a browser test that plays the whole story end to end.

The checker is Google's OR-Tools CP-SAT solver. For each scenario it finds the best possible morning using only the people the plan names for each job, minimising households times minutes late:

$$\min \sum_{r \in \text{routes}} h_r \cdot \text{late}_r$$

A route that can't run at all counts as "not delivered", with a penalty bigger than any lateness, so the screen can say "8 households not delivered" instead of "no solution". A proven zero is a tick, a proven loss is a cross, and if the solver runs out of time it shows a question mark, which never counts as a pass. A second pass picks the earliest sensible morning, so the timeline looks like a real Saturday, and the screen calls it "one morning that gets this result", because other mornings can tie. The same plan always gives the same results.

The gate between the AI and the checker was my learning goal. Nothing the AI returns reaches the screen until it passes a strict format check, has no duplicate or unknown names, and passes plain rules. Then a person confirms every fact with its source line in view, and the server checks everything again before the checker runs. Fix suggestions go through the same gate. One of the AI's suggestions pointed at vehicles where routes were needed, and the gate marked it "Couldn't apply" instead of guessing.

The AI. Gemini (gemini-3.8-flash) reads the plan, with Groq (gpt-oss-120b) as a fallback. I chose between them with a bake-off: three readings each, scored by whether the AI's reading produced the same 100 results as my hand-checked map. Gemini matched all 100 in every run, and Groq matched 55. Later I made the AI cite the exact line for every fact. Both providers kept timing out during that comparison, so neither finished three runs; I kept Gemini as primary on the last complete result and say so in the repo. When Gemini was overloaded, the app fell back to Groq, which is where the example's fix suggestions came from. The example ships with that real, unedited AI output, so anyone can try it without a key.

The interface is meant to feel like a coordinator's run sheet, not a tech dashboard: warm paper, ink, IBM Plex, and ticks and crosses drawn like pen strokes, so the grid works for colourblind viewers too. I designed it with Claude's design tool and built everything in React and plain CSS, with a hand-drawn SVG grid and timeline.

The skills ran in three coding agents: Kiro, then Antigravity, then Codex. Each switch happened when a usage limit ran out, and nothing was lost because the plans lived in files, not in a chat. I also used Claude in a separate chat as a reviewer before approving each step.

Challenges I ran into

My own demo plan was impossible. I designed the example to show exactly two hidden pairs. Before any screen existed, the checker's logic proved that couldn't happen: "doesn't show up" is a stronger version of "leaves early", so if one pair breaks, the other has to break too. My plan looked right and was wrong, which is exactly what this tool exists to catch. The example now has three hidden pairs, and the tests lock them in.

"Late" wasn't always possible. I wanted the demo to say "40 minutes late". It turns out a route can only be late if something frees up later. Otherwise nobody can run it at all. I chose an example where every number on screen is true.

My first accuracy score measured the wrong thing. The bake-off first compared every detail of every schedule. Many mornings are equally good and the solver just picks one, so reordering a list changed 52 timelines but not a single verdict. Scoring what matters (does the plan break, how badly, and for whom) changed the picture completely.

The AI's fixes didn't finish the job. None of its suggestions fixed Joel's key or the truck and Ravi pair. Asking again until it got lucky would have been fishing, and faking a suggestion wasn't an option. So I added "Test your own change": the coordinator's own fixes face the same checker as the AI's.

Trusting my own build. A full audit of the repository found that the server would accept a plan nobody had finished confirming, and that some facts didn't update after a fix. Both are fixed and covered by tests now.

Accomplishments that I'm proud of

The key moment happens on screen, by proof: the AI suggested 12 fixes and the checker turned down 10, each with a plain reason.
Every number in this story can be reproduced with one command from the repo.
One finding surprised me: if Maya stays just 10 minutes longer, until 10:55, all three hidden pairs disappear. At 10:50 they come back. Nobody could see that by reading the plan.

What I learned

Check AI output in layers, not in one place. A strict format, plain rules, a person confirming each fact next to its source, and a server that checks again. No single layer is enough on its own.
Measure what matters. My first AI scores punished differences that changed nothing. Scoring the verdicts instead showed which model actually understood the plan.
Plans need testing, including mine. The checker proving my own demo plan impossible taught me more than anything else in this build.
Ask whether a recommendation was actually checked. Early on, an AI advisor recommended my whole stack. I asked whether it had checked current documentation. It hadn't, and checking changed real choices.
Planning first makes the agent replaceable. Because the plans lived in files, I switched coding agents twice and lost nothing.

What's next for Failure Compiler

Testing it with a real volunteer coordinator. Then a second kind of plan, setting up an event, on the same checker; AI-suggested problem types that run only after you approve them; a minimum-buffer setting; and three problems at once.