Parallax

Gives Codex a visual history of a website : it shows what changed, lets Codex try layout ideas on images before coding, and checks the same steps again to prove the fix worked safely across versions.

Comment

Inspiration

Codex can change frontend code very quickly, but it does not automatically remember how a website looked and behaved before and after those changes.

A small CSS or layout change can make a page overflow, block a button, hide a control, or break a real user flow even when the code still looks correct.

We built Parallax to give Codex a visual history of the website, focused evidence about what changed, and a way to check whether a repair really worked.

What it does

Parallax records a real user workflow in Chromium using Playwright.

For example, a workflow can:

  • open a page;
  • open a menu;
  • click a submenu;
  • close the menu.

After every action, Parallax saves:

  • a screenshot;
  • whether the action passed or failed;
  • the browser size;
  • page width and horizontal overflow;
  • console and page errors;
  • the button or element Codex intended to click;
  • the element that actually received the click;
  • useful DOM and page-layout information.

Each complete run becomes a visual revision.

Parallax can then compare two revisions and:

  • find the first step where something changed;
  • find the first failed action;
  • create image-difference heatmaps;
  • detect the important changed areas;
  • connect those changes to the action that happened just before them;
  • label changes as expected, suspicious, harmful, or uncertain;
  • give Codex a small, focused evidence package instead of every screenshot and every DOM element.

Codex can use that evidence to inspect the relevant source files and make a focused repair.

Parallax then runs the original workflow again without changing the user journey and produces a factual before-and-after report to show whether the fix worked.

Parallax also includes:

  • interface detection: it looks at the source code and DOM structure to understand whether a page is more like a streaming app, online store, dashboard, or an unknown type;
  • visual baselines: it saves healthy versions of the website so later changes can be compared with a known good version;
  • page-region mapping: it identifies important areas such as the header, navigation, main content, sidebar, cards, modal, and footer;
  • layout sandbox: it divides a screenshot into a grid and lets Codex move, copy, delete, or replace image sections to preview a layout idea before changing the real frontend code;
  • visual investigation tools: Codex can request focused reports for overflow, blocked clicks, heatmaps, keyframes, before-and-after views, tracked elements, and visual narratives;

How we built it

Parallax was built with:

  • Next.js and React for the public product website and technical dashboard;
  • TypeScript and Node.js for the main engine;
  • Playwright and Chromium for running browser workflows;
  • Pixelmatch for image comparison and heatmap generation;
  • JSON and PNG files for storing visual revisions and evidence;
  • MCP so Codex can request Parallax tools directly;
  • Railway for the public read-only judge demo.

How Codex and GPT-5.6 were used

Codex with GPT-5.6 was used throughout the project as the main coding and reasoning partner.

It helped us:

  • inspect the repository and understand the architecture;
  • divide the project into small, testable phases;
  • implement the Playwright workflow runner;
  • build visual revision storage and comparison logic;
  • create heatmaps, attention maps, and evidence reports;
  • design the visual-analysis tools and tool-selection planner;
  • build the visual narrative system;
  • implement interface-context detection;
  • build page-region mapping;
  • create the image-mosaic layout sandbox;
  • diagnose bugs during development;
  • review source changes;
  • write and improve tests;
  • build the guided judge-facing website;
  • prepare the Railway deployment;
  • improve the README and submission documentation.

During the controlled repair demo, GPT-5.6 reads the focused evidence produced by Parallax, inspects the relevant source files, identifies the likely cause, and makes a bounded frontend repair.

GPT-5.6 is not embedded inside Parallax, and Parallax does not call the OpenAI API.

Instead, Parallax runs locally as an MCP developer tool that gives Codex structured visual evidence and tools it can request when needed.

The system has three clear roles.

Parallax

Parallax handles the measurable work:

  • runs browser workflows;
  • captures screenshots and browser facts;
  • stores visual revisions;
  • compares versions;
  • creates heatmaps and attention maps;
  • classifies changes using clear rules;
  • selects focused evidence;
  • replays the workflow;
  • verifies the result.

Codex with GPT-5.6

Codex handles the reasoning and coding work:

  • reads the focused evidence;
  • inspects the relevant source files;
  • identifies the likely cause;
  • changes the code;
  • requests layout previews when needed;
  • checks the result against the user’s intent.

Human

The human still decides:

  • what the intended design should be;
  • whether a layout idea is useful;
  • which healthy visual version should be preferred;
  • whether an important source change should be accepted.

Challenges we ran into

The hardest challenge was separating meaningful visual changes from normal pixel differences.

A basic heatmap can show that pixels changed, but it cannot explain:

  • whether the change was expected;
  • whether it was connected to the previous action;
  • whether it created page overflow;
  • whether it moved outside the viewport;
  • whether another element blocked a click;
  • whether unrelated parts of the page moved.

We solved this by combining:

  • image differences;
  • action-target information;
  • DOM geometry;
  • viewport data;
  • overflow measurements;
  • click-interception evidence;
  • workflow expectations;
  • interface context.

Another challenge was keeping the evidence small and useful.

Sending every screenshot, every DOM node, every console log, and the entire repository would create too much noise.

Parallax therefore chooses the smallest useful evidence set for the problem and gives Codex only the important images, measurements, and reports.

We also had to keep the project honest about its limits.

The current automatic repair demonstration is controlled and focused on a known mobile-menu regression. The image layout sandbox creates a rough visual preview only. It does not directly rewrite the website or guarantee perfect text quality.

Accomplishments that we're proud of

  • Built an end-to-end visual workflow system that records real browser actions and saves each run as a visual revision.
  • Detected harmful UI changes using screenshots, browser facts, heatmaps, attention maps, and click evidence.
  • Gave Codex focused evidence instead of overwhelming it with every screenshot and DOM detail.
  • Completed a controlled repair loop that replayed the original workflow and proved the fix with before-and-after results.
  • Added interface detection, visual baselines, page-region mapping, and an image-based layout sandbox.
  • Exposed Parallax through MCP and built a public guided demo using real saved artifacts.

What we learned

We learned that frontend debugging becomes much more useful when image changes are combined with real browser facts.

Screenshots alone are not enough.

A strong visual-debugging system needs to understand:

  • what action happened;
  • what changed on the screen;
  • what changed in the browser;
  • which element was involved;
  • whether the workflow still works.

We also learned that Codex works better with focused evidence.

Instead of sending everything, Parallax gives Codex the smallest useful set of screenshots, measurements, and reports.

Most importantly, we learned that changing code is not the end of the task.

A repair should only be considered successful after the original user workflow is replayed and the result is measured again.

What's next for Parallax

Next, we want to make Parallax work across more projects and teams by:

  • turning it into a reusable remote MCP service;
  • supporting more browsers, devices, and frontend frameworks;
  • using vision models to match websites with the right UI templates;
  • improving the image sandbox so Codex can test richer layout ideas before coding;
  • comparing the final coded result with the approved visual plan;
  • adding shared visual history, reviews, and CI/CD checks.

The long-term goal is to make Parallax a visual repository and planning layer for coding agents.

Built With

Share this project:

Updates