Repair Console: Approval-Gated Playwright Locator Repair

Repair Console turns a broken Playwright locator into a reviewable repair: it captures failure evidence, proposes one selector change, requires approval, then verifies the target test and full suite.

Comment

Inspiration

Playwright tests often fail for a simple reason: the product UI changes, but a locator in the test still points to an old selector. Fixing that is usually small work, yet it interrupts developers and can become risky when automated tools are allowed to modify code without review.

We wanted to explore a more trustworthy form of AI assistance: one that helps developers understand and repair a narrow failure, while keeping the human in control.

What it does

Repair Console is an approval-gated Playwright locator repair tool. In its bundled browser sandbox, a user can simulate a login-button selector regression without a terminal command.

The console captures the failed selector, a short error message, the source location, and a sanitized DOM snapshot. Qwen proposes one replacement CSS selector with concise evidence. The user reviews the exact diff and must select Approve & rerun before any test file changes.

After approval, Repair Console applies one validated selector change, reruns the affected test, and then verifies the full demo suite.

How we built it

We built the project with React, Vite, Express, TypeScript, Playwright, Vitest, Zod, and a server-only Qwen integration.

Codex and GPT-5.6 helped us refine the idea into a deliberately narrow product scope, write the specification and task plan, implement the approval-gated workflow, create tests, and improve the browser demo experience.

The backend validates every proposal and restricts writes to one CSS-selector literal in the Playwright test directory. Server-sent events keep the dashboard timeline updated while the repair is being verified.

Challenges we ran into

The main challenge was making the demo feel like a real customer workflow without making unsafe claims about autonomous code repair.

We replaced terminal-driven mutation steps with visible browser controls, while keeping the mutation limited to the bundled sandbox fixture. We also found and corrected a baseline-state mismatch between the fixture, test selector, and sandbox labels. This reinforced why deterministic browser and mutation tests are essential for a live demo.

Another challenge was balancing live AI behavior with demo reliability. Qwen is the primary provider when configured, while a clearly labelled offline fixture fallback preserves the same approval and verification safety model for rehearsal.

Accomplishments that we're proud of

We created a complete browser-operated repair loop: simulate a regression, inspect evidence, review one diff, approve the patch, and watch both the target test and full suite pass.

We are especially proud that the system does not patch before approval, does not expose credentials to the browser, and restores the test file if verification fails. The product makes its limits visible instead of presenting itself as a general autonomous test-healing system.

What we learned

AI-assisted developer tools become more credible when their scope is narrow, their evidence is visible, and their actions are reversible.

We learned that the most valuable automation is not necessarily fully autonomous. A small, fast, reviewable repair loop can reduce maintenance work while preserving developer judgment and trust.

What's next for Repair Console: Approval-Gated Playwright Locator Repair

Next, we would validate the workflow with real frontend and QA engineers, improve selector confidence signals, and support more controlled locator patterns.

Longer term, we would explore secure repository connections, isolated workers, persistent audit history, and CI integrations. Those additions would require strong authentication, repository isolation, and policy controls before the product could responsibly move beyond its local sandbox.

Built With

Share this project:

Updates