Proofloop

A browser-native debugging tool that captures a bug, freezes it as a behavioral test, and lets Codex diagnose, patch, and prove the fix against the exact same test.

Comment

Inspiration

Bug reports usually describe a symptom, not the exact behavior that produced it. Developers must reconstruct the sequence, decide what “fixed” means, and then trust that the test used to verify the patch still represents the original problem.

Coding agents can accelerate the patch, but that makes independent verification even more important. We built Proofloop around one rule:

The agent may change the application, but it may not change the proof.

What it does

Proofloop turns browser bug evidence into a protected, executable proof of behavior.

A developer records the failing workflow through a Chrome extension side panel. Proofloop captures interaction evidence, redacts sensitive information, and lets the developer review it before anything is sent to a model.

With explicit consent, GPT-5.6 interprets that evidence through the pinned Codex CLI and returns a schema validated reproduction plan. Proofloop then compiles the plan into a deterministic Playwright test, records its exact hash, and freezes it.

The proof is first executed against the original application and must fail for the intended behavioral reason. Codex then diagnoses the problem and creates a candidate patch inside a disposable Git worktree. Proofloop runs the exact same frozen proof against that candidate and executes the relevant regression tests.

A fix is marked verified only when:

  • The original application failed the proof.
  • The unchanged proof passed against the candidate.
  • The regression suite passed.
  • The temporary runtime and worktree were cleaned up successfully.

The developer can inspect the exact candidate diff and explicitly apply only the verified patch to an unchanged local repository.

How we built it

Proofloop is a local-first pnpm workspace built with Node.js 24 and strict TypeScript.

The product consists of a React and Vite Chrome extension, a loopback-only Fastify runner, a React demo application, shared Zod contracts, a privacy focused redaction package, and a Playwright proof compiler.

GPT-5.6 is used only for structured interpretation after explicit user consent. Codex uses the developer’s existing ChatGPT authentication rather than accepting API keys through the extension. Model output is schema validated and never executed as generated test code.

Codex also played a major role in building Proofloop. It helped implement the application, tests, recovery paths, security boundaries, packaging, and documentation. We made the core product and engineering decisions: freezing the proof before diagnosis, requiring explicit consent for model and repository operations, isolating candidate changes, and refusing to treat infrastructure failures as verified fixes.

Challenges we ran into

Browser evidence also crosses several trust boundaries. We had to redact sensitive values before model use, limit what the extension could capture, keep the runner bound to localhost, and prevent the browser from supplying arbitrary paths, commands, prompts, models, or credentials.

Recovery was another major challenge. Reloads, runner restarts, failed candidate processes, occupied ports, and partial Git operations all needed to fail closed without silently retrying model calls or mutation.

Accomplishments that we're proud of

We completed a working end-to-end proof loop that captures a deterministic browser bug, generates a structured plan with GPT-5.6, confirms the original failure, lets Codex create an isolated candidate, and verifies that candidate with the unchanged proof and regression suite.

We are especially proud of the exact-byte proof identity, tamper-evident artifacts, reviewable candidate diff, explicit consent gates, disposable worktree lifecycle, and the distinction between “verified in isolation” and “applied to the repository.”

What we learned

We learned that AI is most useful when its judgment is separated from deterministic authority. GPT-5.6 is good at interpreting human behavior, and Codex is good at navigating and modifying a codebase, but deterministic software should decide what executes and whether the evidence still matches.

We also learned that cleanup and recovery are part of correctness. A passing assertion is not sufficient if the candidate used the wrong source revision, the proof changed, regressions never ran, or temporary processes were left behind.

What's next for Proofloop

Next, we want to support more browser behaviors and application structures while preserving the same proof invariant. We also want richer privacy controlled evidence, broader framework support, and a reviewed pull request handoff for verified patches.

The long term goal is to make the path from “I found a bug” to “this exact behavior is now proven fixed” more trustworthy.

Built With

Share this project:

Updates