Faultline — The Adaptive Debugging Gym

Inspiration

Every developer learns to code from tutorials — but nobody teaches you to debug. You learn it the hard way, alone, at 2 AM. Meanwhile, AI assistants have swung the other way: they just fix your bug for you, and you learn nothing. We wanted the middle path: an AI that watches you debug and coaches like a great senior engineer — one who asks the right question instead of grabbing your keyboard. Think LeetCode for debugging, minus the DSA grind.

## What it does

Faultline is an adaptive debugging gym. You get a real, broken TypeScript application in a Monaco editor. You investigate, edit, run tests, and submit hypotheses about the cause. An AI instructor — GPT-5.6 running through the Codex SDK — receives the evidence of how you work (your diffs, run results, and hypotheses), and responds with Socratic hints that must cite that evidence. It never spoils the fix. A three-level hint ladder escalates only when you're stuck: question → direction → explanation.

The kicker: the coach never grades you. A deterministic, behavior-based grader runs hidden probes against your fix — and accepts any behaviorally-correct solution, not just the reference patch. Fake fixes that patch symptoms fail. Genuinely different valid strategies pass. An observability dashboard shows everything the coach was given, what it inferred about your misconceptions, and how its teaching adapted.

## How we built it

• Three hardened containers: a Next.js app (UI + SQLite event journal), a coach service wrapping the Codex SDK (fresh GPT-5.6 thread per coaching turn, structured JSON output), and a secretless runner that executes learner code with no network and no credentials.
• Evidence pipeline: every save, run, and hypothesis becomes a typed event; the coach's every claim must cite event IDs — no vibes-based tutoring.
• Codex built Faultline, and Codex runs inside it. Primary development runs through a single Codex (GPT-5.6) session, with a team of specialized Codex agents handling architecture review, SDK verification, and lab/QA design. The full collaboration transcript backs our "where Codex accelerated us" docs.

## Challenges

• Designing an isolation model where learner code can never reach the AI credentials or the hidden grading probes — this forced a full architecture revision mid-design (our Codex review team caught it).
• Getting trustworthy structured output from an agentic SDK: the model produces only the coaching payload; every trusted field is copied by the service, never generated.
• Building a grader that proves understanding: each lab ships with two intentionally different correct fixes and eight near-misses that must fail — validated before any UI existed.

## What we learned

Multi-agent AI collaboration works best adversarially: our biggest design flaws were caught by AI reviewers instructed to reject, not agree. And "the AI can see everything you did" turns out to be the whole product — evidence-grounded teaching feels completely different from chat.

Built With

• docker
• docker-compose
• fastify
• gpt-5.6
• monaco-editor
• next.js
• node.js
• openai-codex-sdk
• playwright
• pnpm
• react
• sqlite
• typescript
• vitest
• zod