Irus

• 

  Snapshot of Landing Dashboard

Inspiration

"You take the backend, I'll take the frontend." Everyone has said it. Here is how it ends.

You build POST /api/checkout. It accepts JSON with email and an integer amount. Your tests pass.

Your teammate builds the form. It sends multipart/form-data with user_email and total as a string. Their tests pass.

Git merges both branches without a conflict, because you edited different files. The app is broken and every signal says it is fine. Your branch was tested against base. Their branch was tested against base. CI verified ( A \cup \text{base} ) and ( B \cup \text{base} ), and never once built ( A \cup B ), which is the only state where the bug lives. Neither of you was wrong about your own half, so neither of you owns the failure. It clears code review and gets found in staging, or by a user.

Then we handed the same split to two coding agents and watched it happen in four minutes instead of four days. An agent finishes its half and reports done, and the report is true. Give ( n ) agents one surface each and the boundaries between them grow like ( \binom{n}{2} ), while the tests each agent writes still cover one side.

What it does

Irus typechecks that boundary. It reads both halves, compares what each one declares, and names the disagreement with both file paths.

$ irus check POST /api/checkout endpoint exists PASS route mounted PASS client calls it PASS encoding matches FAIL client sends multipart, server expects json payload shape matches FAIL server requires amount (int); server requires email (str)

5 finding(s) introduced this session (3 high) | baseline acab5370f983 suppressed 1 pre-existing $ echo $? 1

Then it proves the finding by running it. Irus builds the request your frontend would actually send, pushes it through the framework's own test client inside a transaction that rolls back, and reports the 422 that the application returns. The verdict comes from your app, not from our checker, which is what makes it evidence instead of an opinion.

It baselines against the git merge-base of the active worktrees, never against wall-clock time. A repository's pre-existing weirdness stays invisible and you see what this session introduced. Anchoring to "when the tool started" would hide the bug in the common case where the agents were already working before you launched it.

One person adopts it alone. No spec, no contract tests, no buy-in from your teammate. It runs as a merge gate, a CI step, or an MCP server that agents query as text.

How we built it

The engine is Python standard library and nothing else. Zero required dependencies, no network, no build step, and no model anywhere in the verification path, so the answer is identical every run. Python gets parsed with ast. TypeScript gets a dependency-free scanner that masks strings and comments before any structural parsing, so a brace inside a string cannot fool it.

Proof runs in three tiers, cheapest first: validate the assembled request against the handler's declared schema without sending it, drive the real handler in-process through the framework test client, then real HTTP for safe methods only. Nothing leaves localhost, no mutating method crosses the wire, and we fingerprint the local store before and after every proof run. A change aborts loudly instead of passing quietly.

Watch mode draws the map. In a normal dependency graph you care about the edges that exist. Here you care about the edge that should exist and does not, and absence renders as nothing at all, so we draw it as a dashed red arc with the two ends pulled apart.

167 tests pass. A sweep of dispatch covers 876 files in 1.59 seconds; redash, 861 files in 1.08 seconds; reflex, 894 in 2.66. Re-checking after one edit takes 83 ms.

Challenges we ran into

We tested the premise instead of assuming it. Irus has two stages because we did not know whether a half-built repository would even boot. So we wrote down predictions for five realistic mid-build states, then ran them. Three booted, two did not, and all five matched the prediction. missing_import died on ModuleNotFoundError, truncated_edit died on SyntaxError: '(' was never closed. Execution proof is available often enough to be worth building and absent often enough that a static stage has to work alone.

Our headline demo failed the first time we ran it for real. We gave two agents one half each of a three-feature prose spec, in separate worktrees, with no knowledge of each other. Git merged both branches with zero conflicts, and one seam diverged: the backend agent wrote marketing_emails, the frontend agent wrote marketing_opt_in. Irus found nothing. Every route reported "client absent."

Four defects, each visible only because an agent wrote the code instead of us:

1. Neither agent wrote an inline fetch. The frontend agent built a shared sendJson(method, path, body) helper, so the whole contract lived at the call site. Our scanner read fetch( and stopped. It now follows one hop through a project-local wrapper, and one hop on purpose, since deeper chains multiply the odds of attaching a body to the wrong request.
2. Every request looked like a GET. The wrapper used ES6 shorthand { method, headers } and we only matched method: method.
3. Every endpoint looked bodyless. The Pydantic models sat in schemas.py and the handlers in routers/, and our model registry was per-file. Every real FastAPI project splits those.
4. A patch script reported success while changing nothing. We caught it because the behaviour did not move.

Without those fixes the answer would have been a false negative on the exact bug the tool exists to find.

Measuring precision honestly cost us the prettier number. We ran Irus unfiltered across five repositories nobody here wrote and published every finding, including the wrong ones. 314 findings, 226 of them judgeable, 72.1% precision. The orphan_endpoint check sits at 18% and drags the total down. We could raise it by matching paths the way the adjudicator judges them, which would push the published number toward 100% by construction and measure nothing, so we left the check weak, reported it at low confidence, and shipped the 72.1%.

What we learned

A tool that publishes only its hits cannot be measured. Precision is the number people should trust least when the vendor picks the examples. So the ledger is a command: irus ledger ../repo-a ../repo-b reproduces our table on your repositories, false positives included.

Say what you cannot do, in the README, above the fold. Client indirection follows one hop, so recall on heavily abstracted code drops and we have not quantified the gap. A body assembled through a chain we cannot follow reports as unknown rather than as a guess, because that is our limitation and not your defect. A spread (...rest) suppresses missing-field checks for that seam.

Our own pitch was overstated, and the experiment caught it. We claimed both agents finish and report success, so the gap belongs to nobody. That is not what happened. The frontend agent listed every field it had invented, said plainly that none of the contract came from the spec, put the odds of all three features interoperating below 20%, and asked for a shared contract. The mismatch still merged clean and still slipped past every automated check. The honest claim is that the bug survives the merge and the whole toolchain, not that nobody suspects it.

One mismatch across three features, from one session, run by two Claude subagents on one machine. Enough to prove the tool catches something real. Nowhere near enough to estimate a rate, and the vendor diversity is missing. Two of the three seams matched exactly, including a multipart part name the agents agreed on with no coordination.

Built With

• html5
• python
• render