Proofline — Evidence-based review for AI-generated code

Inspiration

AI coding agents make it easy to generate changes quickly, but speed creates a new problem: how do developers know an AI-authored change is actually safe to merge? Code review alone often misses broken keyboard flows, accessibility regressions, visual drift, and user journeys that were never tested. Proofline was inspired by the need to turn “looks good to me” into real, reproducible evidence.

What it does

Proofline is an evidence-based review layer for AI-generated code. It compares Git refs, identifies changed files, selects the user journeys most affected by those changes, and runs fresh browser audits against local baseline and candidate versions.

It captures screenshots, visual diffs, accessibility findings, console errors, Playwright traces, retry confidence, and a merge verdict: Blocked, Needs Review, or Ready to Merge. It can record a local user journey, retain approved visual baselines, export Markdown/JSON/PR-ready evidence, and post a verdict plus annotations through GitHub Actions.

How we built it

We built Proofline as a local-first Node.js application using Express for the dashboard and audit API. Playwright runs isolated browser journeys and captures traces/screenshots; Axe scans for accessibility regressions; pixelmatch and pngjs produce visual-difference overlays.

Proofline uses local Git diffs and detached worktrees to compare a baseline commit with an AI-authored candidate safely. A proofline.json configuration maps source files to user journeys, enabling impact-aware test selection. We also added a CLI and GitHub Actions workflow to produce PR comments and workflow annotations.

Codex and GPT-5.6 were used throughout the project: architecture, implementation, debugging, UX refinement, test design, documentation, and creating evidence-backed repair briefs.

Challenges we ran into

The hardest challenge was making the product feel like a real developer tool instead of a static dashboard. We had to orchestrate real Git worktrees, start separate local app versions, run browser checks reliably, and preserve evidence artifacts.

Another challenge was balancing strong automation with trustworthy results. We added fresh-browser retries so Proofline distinguishes reproducible failures from possibly flaky tests. We also kept the project local-first: repositories, screenshots, traces, and diffs remain on the developer’s machine.

Accomplishments that we're proud of

Built a complete real-Git demo, not hardcoded metadata.
Detects a realistic regression where a semantic checkout button becomes a clickable div, breaking keyboard completion.
Runs impacted journeys rather than blindly running every test.
Produces screenshots, visual overlays, accessibility evidence, console evidence, and Playwright traces.
Includes a repair-and-reverify loop that proves the fix in a fresh browser run.
Records local journeys without capturing typed values.
Creates PR-ready verdicts, GitHub Actions comments, and changed-file annotations.
Retains approved visual baseline history locally.

What we learned

We learned that trustworthy AI developer tools need to explain their reasoning, not merely return a pass/fail result. Developers need to know what changed, which journey was selected, what evidence was collected, and why a merge was blocked or allowed.

We also learned that local-first design is a major trust advantage for developer tooling. Keeping source code and browser evidence on the developer’s machine makes Proofline easier to adopt for sensitive repositories.

What's next for Proofline — Evidence-based review for AI-generated code

Next, we want to add richer journey recording, broader framework support, and native mobile testing. We also plan to improve GitHub integration with deeper PR annotations, reviewer baseline approval history, and organization-level quality policies.

Longer term, Proofline can become the verification layer between AI coding agents and production: AI generates a change, Proofline gathers evidence, a developer reviews the verdict, and only verified changes move forward.