Doppel

Inspiration

Design systems don't fail at adoption. They fail at entropy. A team ships a component library, everyone uses it for a while, then a deadline hits and someone hand-rolls a because the library's Button doesn't have the exact variant they need. Someone copies that button. A contractor who never read the docs builds three more. Two years later the library has one Button and the app has forty-one others, and nobody can tell you where they are.

This isn't cosmetic. A rebrand updates the library and misses the forty-one. The library's Button handles focus rings, aria-disabled, and keyboard activation; the hand-rolled copies usually don't, which is the single most common source of accessibility-audit findings in regulated frontends. Every hand-rolled copy also has to be maintained, tested, and reviewed on its own from then on.

Existing tooling doesn't solve this. ESLint can ban a raw button but can't tell you which library component should replace it. Design-token linters check colors, not structure. Codemods handle known-shape mechanical renames, not "this bespoke div soup is semantically a Card." We wanted to build something nobody builds at an AI hackathon: frontend governance tooling that understands the problem well enough to know when a bespoke component should stay as it is.

What it does

Doppel is a design-system drift agent. Point it at a repo and it indexes the design system's own components, props, variants, and source, using AST analysis. It extracts drift candidates by walking every file, with zero model calls: pure deterministic AST analysis plus a rule-based shortlist that cuts inference volume by an order of magnitude before any LLM sees anything. It triages candidates with a small, cheap model, batched. It adjudicates each one into real DRIFT, a deliberate INTENTIONAL_DEVIATION (a marketing hero, a one-off illustration), or NOT_DRIFT (a layout wrapper that only looks like a card). For genuine drift, it writes the replacement JSX and a prop mapping, and it flags anything the design system component can't express instead of silently dropping a gradient or a custom style.

Then it verifies every migration by actually rendering it: original and patched, side by side, inside a real sandbox, screenshotted at multiple viewports, pixel-diffed, typechecked, and linted. That lands one of five verdicts: PASS, REVIEW (a real but plausible visible difference), FAIL, UNVERIFIABLE (the harness genuinely couldn't mount the candidate in isolation, reported honestly instead of guessed at), or INTENTIONAL_DEVIATION.

Finally it reports all of this in a dashboard: a drift map, finding cards with before, after, and diff screenshots, a cost panel showing which model tier handled which task and what the routing saved compared to running everything on the largest model, and a one-click PR export for anything that passed.

Doppel never proposes a patch it hasn't rendered. That's the rule the whole project is built around.

How we built it

Everything is TypeScript, in a pnpm monorepo. A deterministic scanner does ts-morph AST analysis with no network or model calls, tested against a fixture we built with 18 known drift cases planted by hand. An agent layer routes every model call through one function. A verifier package generates an isolation render harness and drives it inside Nebius Token Factory Sandboxes. A Next.js dashboard sits on top, backed by Postgres.

Model routing uses three Nemotron tiers for three different jobs. A small model batch-classifies cheap candidates. A mid-tier model makes the real judgment call, drift versus intentional versus not drift, with full source context. The largest tier is reserved for the two genuinely generative tasks, writing migrations and repairing a failed one, and it never runs in a per-candidate loop for anything cheaper than that. Every call gets logged with its token count and cost, so the cost panel's numbers come from real data instead of an estimate.

Verification is where most of the engineering went. One warm sandbox image gets built per run, with Node, Playwright, and Chromium installed once. Every candidate then forks from that same checkpoint instead of paying the install cost again. Each render gets screenshotted at three viewports, pixel-diffed, and type checked with an in-memory tsc, all before a verdict shows up anywhere.

Challenges we ran into

Most of the debugging time went into two places: making structured LLM output trustworthy, and getting a React app to render correctly inside an ephemeral sandbox.

On the model side, we spent a while chasing what looked like a truncated JSON string bug. It turned out the model was reliably failing at one specific task: escaping a double quote inside a JSON string field, even under a strict schema. Switching generated JSX to single-quoted attributes made that failure impossible and took our worst-performing test case from 3 passing migrations out of 15 to 15 out of 15. Separately, reasoning models sometimes returned null content when a small token budget got used up entirely by internal reasoning, with nothing in the response to tell you that apart from the model simply choosing to answer with nothing.

On the sandbox side, large stdout silently switched encoding from ASCII to base64 above some size threshold that isn't documented anywhere. That cost us two failures that looked unrelated until we manually decoded a captured log and found the pattern. Chromium also blocks ES module scripts loaded over file://, which broke our first attempt at screenshotting a built site until we served it over a local HTTP server instead. The subtlest bug came last: React 18 and 19's createRoot reports an uncaught render error asynchronously instead of throwing it back out of render(). Our harness wrapped render() in a try/catch, which never saw these errors, so a candidate that genuinely couldn't render in isolation came back as FAIL with a fabricated 0 percent pixel delta on a blank screenshot, instead of the honest UNVERIFIABLE the system is supposed to report. We found this during final review, fixed it, and put the real, less flattering numbers back into our own README instead of leaving the old ones in place.

Deploying to a serverless platform brought its own bug: file tracing only bundles files that get statically imported, not files read through a runtime filesystem call. Our fixture repo and our own prompt files quietly didn't ship with the deployed function until we tracked both down by hand.

Accomplishments that we're proud of

We hit 100 percent recall and 100 percent precision on our fixture's 18 planted drift cases, checked against a committed ground-truth file rather than just claimed. The verification loop genuinely tells PASS from REVIEW from an honest UNVERIFIABLE, instead of three ways of saying the same thing. The cost panel shows real routing math: roughly half the spend saved compared to running everything on the largest model, computed live from logged token counts on every run. We ran Doppel on its own dashboard and it correctly found two real drift instances we hadn't noticed in our own code. Near the end, when we noticed our own README's "15 out of 15 passed typecheck" claim no longer matched what the full pipeline actually does, we measured the real number and published that instead of leaving the better-looking one up.

What we learned

The hard part of having an LLM migrate your code was never writing the migration. It's trusting the result enough to merge it, and nothing but actually rendering the output earns that trust. Structured-output guarantees are narrower than they sound: a strict schema guarantees valid JSON shape, not that a string field's content is complete. A sandbox's real job is less "run my code" and more "tell me honestly whether my code can run at all," and getting that right takes real engineering, not just a render call. An honest "I don't know" verdict, used when it's actually true, is worth more to a tool like this than a confident guess.

What's next for Doppel

We want a real background job queue so verification on a large repo isn't bounded by a single request's time limit. We also want to close the migration-quality gap we found and reported rather than hid: migrations sometimes skip importing the component they just wrote JSX for. Beyond that, we'd like to support a second design system once coverage of this one is solid, and run Doppel against real production codebases instead of just our own dashboard and fixture, since that's the only way a precision number earns real trust.

Built With

• chromium
• contree
• drizzle-orm
• github
• nebius-token-factory
• nebius-token-factory-sandboxes
• neon
• next.js
• node.js
• nvidia-nemotron
• openai-sdk
• pixelmatch
• playwright
• pnpm
• postgresql
• react
• rest
• shadcn/ui
• tailwind-css
• ts-morph
• typescript
• vercel
• vite
• zod