PixelBisect

Visual testing catches tomorrow’s regressions. PixelBisect finds the exact commit that caused yesterday’s.

Comment

Inspiration

Visual-regression tests can tell developers that an interface is broken today, but they rarely answer the harder question: which historical commit introduced the problem?

Finding that commit often means manually rebuilding old revisions, comparing screenshots, reading changelogs, and searching through CSS changes. I built PixelBisect to automate that investigation.

Visual testing catches tomorrow's regressions. PixelBisect finds yesterday's.

What it does

PixelBisect is a local visual-regression forensics CLI.

A developer provides a local Git repository, a known-good commit, a known-bad commit, a page URL, and a CSS selector. PixelBisect then:

  1. Creates an isolated Git worktree without modifying the developer's active checkout.
  2. Verifies that the good and bad endpoints are visually different.
  3. Uses native git bisect run to search the commit history.
  4. Installs dependencies, starts the application, and captures the selected interface at each tested commit.
  5. Classifies every tested commit as GOOD or BAD using deterministic screenshots and thresholded pixel comparison.
  6. Identifies the exact first-bad commit.
  7. Produces a self-contained HTML evidence report.

The final report includes:

  • Culprit commit hash, author, date, and message
  • Last-good and first-bad screenshots
  • An interactive before/after slider
  • A highlighted pixel-difference image
  • Changed-pixel statistics
  • Changed computed CSS properties
  • Every commit tested during the bisect
  • The Git patch that introduced the regression

All screenshots, styles, and scripts are embedded in one offline HTML file. PixelBisect does not upload repository code or screenshots and requires no account, API key, database, hosted backend, or paid service.

The demo

The reproducible demo generates a fleet-operations dashboard with a deterministic 64-commit history.

The dashboard still reports 18 drivers online, but seven driver markers have silently disappeared underneath service-zone overlays. The data and DOM remain correct, making the bug subtle and difficult to trace.

PixelBisect searches the 64-commit range in six midpoint comparisons. It identifies a design-token refactor that changed only:

- --layer-map-marker: 30;
+ --layer-map-marker: 3;

That one-line change moved the driver markers underneath the map overlays.

The final comparison detects 1,589 changed pixels out of 778,800—approximately 0.204%—and connects that small visual symptom directly to the responsible commit and source patch.

How I built it

PixelBisect is written in TypeScript and runs on Node.js.

It uses:

  • Git for revision resolution, detached worktrees, first-parent history, and native binary search
  • Playwright and Chromium for browser automation and deterministic screenshots
  • pixelmatch and pngjs for exact-dimension image comparison
  • HTML, CSS, and JavaScript for the portable evidence report
  • Vite for the generated demonstration application
  • Node's built-in test runner for unit, integration, and end-to-end tests
  • GitHub Actions for Windows and Ubuntu verification

Historical application commands run inside a detached worktree. This protects the developer's current branch and uncommitted files while PixelBisect examines older revisions.

At every selected commit, PixelBisect installs or reuses dependencies, optionally builds the project, starts its server, waits for readiness, captures the configured element, compares it with the known-good baseline, and displays live progress.

How I used Codex and GPT-5.6

Codex with GPT-5.6 was my primary development partner throughout the project.

Codex helped me:

  • Turn the initial idea into a focused developer-tool MVP
  • Design the CLI configuration and investigation workflow
  • Implement the TypeScript codebase
  • Integrate Git bisect, worktrees, Playwright, and pixel comparison
  • Design and refine the offline HTML report
  • Generate the deterministic 64-commit demonstration repository
  • Create adversarial tests for occupied ports, failed builds, hanging commands, invalid configuration, and interrupted investigations
  • Analyze cross-platform CI logs and diagnose Windows process-tree races
  • Improve the fleet dashboard and report through rendered visual inspection
  • Package and verify the judge-ready release

Several important product decisions came from this collaboration:

  • Use native git bisect run instead of implementing a custom search algorithm
  • Keep historical builds away from the developer's active checkout
  • Treat interruption cleanup as a product feature
  • Keep all repository evidence and screenshots local
  • Build a narrow, reliable MVP instead of claiming unsupported functionality
  • Make the offline evidence report the product's primary visual interface

Built With

Share this project:

Updates