Replay

Logo
player (3)
player (2)
player (1)

Inspiration

I work on a lot of bugs, I get bug reports, I read them, sometimes they are missing a reproduction video, sometimes the reproduction video is ambiguous by itself.

I usually ask my coding agent to reproduce the bug, but it prefers to reproduce it in units. When verified against the actual app the reproduction is invalid. This is almost always the case for big apps, it is simply too complex for LLMs to reproduce it in contained units.

So I find myself asking my coding agent to use Playwright to drive a browser for an E2E reproduction.

But it is painfully slow! The coding agent thinks a lot between steps, you have to babysit it throughout the process.

So I thought to myself, what if we had a recording of the actual reproduction that the coding agent did, but without the overlaps?

Then I took it another step and asked: What if we were able to share these recordings with other people?

And this project was born!

What it does

Replay gives your coding agent the power to reproduce and record a bug, the end result is a well formatted reproduction using rrweb that gets right to the point. No noise.

A 10 minute playwright session becomes a 30 session bug reproduction recording.

You can share replays with others. You can use the AI Replay Assistant (GPT 5.6 Terra) to get even more information about the session replay.

How we built it

Replay is a TypeScript pnpm monorepo built around one deliberate separation. Playwright drives the browser, and Replay observes and captures that same browser. It never proxies or reimplements Playwright's tools.

Capture uses rrweb over the Chrome DevTools Protocol to record a DOM-based timeline, not video. That's what makes the artifact tiny, seekable, and portable.
Replay MCP embeds a pinned @playwright/mcp in-process and forwards every browser_* call to it, injecting an optional replay_marker parameter into each tool's schema and stripping it before Playwright ever sees it. So the agent's normal browser workflow stays untouched, but every action is captured with request/response brackets.
A local daemon (bound to 127.0.0.1) owns the Chrome lifecycle and provisions it lazily. Chrome only launches when the agent actually uses the browser, not when the MCP client starts.
The player does idle-aware playback (cut / fast-forward / keep), seeking, and speed controls, and projects the assistant's chat timestamps onto the trimmed clock.
Portable .replay artifacts are a versioned bundle with integrity checks. A share server (on Railway) validates uploads and serves hosted replays behind bearer links.
The Ask AI assistant (GPT-5.6 Terra, in Codex) answers questions about a replay and highlights elements on screen as it explains.
Everything ships as a packaged macOS runtime that bundles its own Node. One curl | sh wires up the MCP server and the /replay-browser-capture skill.

Challenges we ran into

The browser rendezvous. Capturing the exact Chrome that Playwright drives, without proxying Playwright or making the user manage ports, was the hard core of the project. We solved it with an in-process bridge and lazy CDP attach so Chrome stays closed until the browser is genuinely used.
Atomic markers. A marker had to bind to its action even under concurrent tool calls. We capture the checkpoint atomically with the action itself, so association never depends on call ordering.
Trimming without lying. Cutting the agent's dead air while keeping the assistant's chat times aligned meant projecting those timestamps onto the idle-cropped clock instead of the raw one.
Privacy by default. Passwords are masked, and scripts, API responses, and assets over 10 MiB are excluded, without breaking DOM fidelity on playback.

Accomplishments that we're proud of

A minute-long agent run becomes about 9 seconds of signal. The bug, and nothing else.
Zero-touch capture. The agent drives the browser and captures through one server. You never touch Chrome, ports, or capture config.
Reviewers install nothing. One share link opens a DOM-based replay in the browser.
The one thing an agent can't fake: a faithful recording of what it actually did in a real browser, portable enough to hand to anyone.

What we learned

For complex apps, LLMs reproduce bugs badly in isolated units. The reproduction looks right but is invalid against the real app. Driving a real browser end-to-end is the only reliable ground truth.
A recording of the agent's actual run is the antidote to hallucinated repros. It can't be faked after the fact.

What's next for Replay

Agent-readable replays (Phase 4B): a scoped query API over the summary layer plus MCP tools, so another coding agent can interrogate a shared replay directly instead of a human re-watching it.
Production hardening (Phase 5): rapid navigation and SPA routes, frames, uploads, canvas, long replays, storage recovery, and accessibility.
A richer masking policy so Replay is safe to point at production apps out of the box.
Authentication and organization members. Move beyond public bearer links to authenticated access, with multi-tenant workspaces where an organization's members can sign in and see the replays shared within their team. This turns share links from open URLs into access-controlled, per-tenant resources.