ripr

• 

Verification became the bottleneck

Last year, I used AI at a scale where verification—not generation—became the bottleneck.

CI and mutation testing were great for keeping agents on the rails, but once you industrialize AI, they become too expensive to run continuously. At one point, I was consuming roughly $40,000 a month of sponsored GitHub Actions capacity.

So I built ripr.

What ripr does

ripr is static mutation-exposure analysis for agentic software development.

It identifies changed behavior that the current tests would fail to notice breaking. Instead of waiting for a long mutation-testing run after code reaches GitHub, ripr brings that evidence into the drafting loop.

The result can appear directly in the IDE. A developer or coding agent can see where the test evidence is weak, which behavior is missing protection, and what focused repair is needed before the pull request is opened.

What changed during OpenAI Build Week

ripr existed before Build Week as a static mutation-exposure analyzer.

During Build Week, I used Codex with GPT-5.6 to add before-and-after comparison. Before this work, ripr could find weak test evidence. It can now distinguish what the current pull request weakened from test debt that was already there.

A pull request can replace an exact assertion with a generic one while every test remains green. ripr compares the evidence before and after the change and can identify that the current pull request made the test weaker.

The comparison is produced once and reused across review, CI, the IDE, and agent repair briefs. Ambiguous or incomplete comparisons remain unknown rather than being misrepresented as regressions caused by the pull request.

How I built it

Codex with GPT-5.6 was the primary engineering environment for the eligible work.

I used it for repository inspection, implementation, fixtures, integration, testing, and review follow-up across a mature Rust codebase.

The Build Week work added:

1. A canonical producer that compares test evidence at the base and head revisions.
2. An authority boundary that determines what may be attributed to the current pull request.
3. Shared projection of that result into review, CI, editor, and agent surfaces.

I retained the load-bearing decisions:

• compare semantic evidence rather than relying on line proximity;
• produce causal attribution once rather than allowing each output surface to reinterpret it;
• treat incomplete evidence as no authority to blame the pull request; and
• keep the analysis deterministic before handing a bounded repair task to the model.

In the product workflow, ripr establishes the missing evidence first. GPT-5.6 receives a focused repair target instead of a vague request to write more tests.

The model executes the repair. It does not invent the evidence used to justify it.

Challenges

The hard part was attribution.

A weak test near a diff may have been weak for years. Finding a gap at the head revision does not mean the current pull request caused it.

The implementation therefore needed stable evidence identity, explicit base and head states, and a failure-closed unknown state. It also needed every downstream surface to preserve the same result rather than reconstructing causality from paths, line numbers, or wording.

What I learned

Once generation becomes cheap, the important question is no longer whether an agent can produce a plausible change.

It is whether the surrounding system can establish what changed, what evidence protects it, what became weaker, and what the agent is actually justified in repairing.

Deterministic tooling should establish the obligation. The model should perform the bounded work.

What is next

Rust is the mature path today.

The next work is faster incremental comparison, clearer causal results in the IDE and code review, broader language support, and tighter repair receipts.

ripr is static and advisory. It does not execute mutants, prove correctness, or claim that a test suite is adequate.

Built With

• actions
• analysis
• code
• codex
• developer
• git
• github
• gpt-5.6
• language
• mutation
• openai
• protocol
• rust
• server
• static
• studio
• testing
• typescript
• visual