CodeSense

Inspiration

AI generates code faster than most junior engineers can learn to evaluate it. The bottleneck isn't writing code anymore; rather, it's knowing when code is unsafe, what it assumes, and how it might break in production. Traditional CS education covers transactions and idempotency, but not the skill of recognizing when those things matter in an unfamiliar codebase. I built CodeSense to see if an AI-guided experience could help people practice that kind of judgment.

What it does

CodeSense turns an AI-generated pull request into an interactive lesson on engineering judgment. In the hero case (TinyShop), you review a checkout implementation that looks fine but has a real risk: Stripe charges the customer, the DB save fails, and a client retry creates a duplicate charge.

You start by selecting suspicious lines and explaining why. CodeSense adapts based on what you notice: the model interprets your reasoning, but deterministic code picks the coaching path. You then predict how the system will behave, watch the failure happen step by step, add an idempotency key to fix the duplicate charge, and run real tests in the browser. After the tests pass, CodeSense replays the same failure: the duplicate charge is gone, but Stripe and the order DB can still disagree. The lesson ends with a summary grounded in your own words and the evidence you actually saw.

How we built it

CodeSense is an explicit state machine: Review → Coach → Predict → Break → Fix → Verify → Replay → Reflect → Summary.

All learning content (concepts, questions, tests, failure traces, reflection prompts) is authored in scenario files, rather than being generated freely. The model's scope lies in what's genuinely ambiguous: interpreting free-form learner language and extracting supporting quotes. Deterministic TypeScript handles everything consequential - coaching routes, question selection, test execution, and conclusions.

The learner's patch runs in a browser Web Worker, so Verify executes real code. The scenario also includes a hidden test case designed to catch incomplete fixes. I used GPT-5.6 as my architecture collaborator and Codex as my implementation partner. At runtime, I use a smaller model (5.4 nano) for the narrow classification task.

Challenges we ran into

The hardest part was drawing the line between what the model controls and what it doesn't. Letting the model freely generate coaching and grades would've been easier, but it'd also make the learning standard opaque and unreproducible. So I had to build structured model outputs, verbatim-quote guards, deterministic depth calculations, and authored fallbacks.

The most important problem showed up late. The original Fix stage taught an order-state rule, but the failure lesson was about a cross-system payment problem. Each screen made sense alone; the full journey didn't. I worked with Codex (engineer) and GPT 5.6 on the web app (PM) - together we refactored Fix around payment idempotency so the learner actually fixes the bug they just watched happen.

Accomplishments that we're proud of

The post-fix replay is the thing I'm happiest with. After all tests pass, CodeSense injects the same DB failure again. Stripe reuses the original charge (proving the fix worked), but the stored order still says "pending." You don't just hear that passing tests ≠ a fixed system — you watch those two ideas separate.

More broadly: CodeSense is adaptive without being arbitrary. Different learners get different coaching, but the consequential decisions are inspectable and reproducible. The model never silently becomes the source of educational truth.

What we learned

Educational AI shouldn't always try to be maximally helpful. A coach that immediately gives you the answer removes the productive friction - where judgment comes to form. Having learners predict before seeing evidence, then reflect after changing code, is fundamentally different from chatting with an answer bot. I also learned that product coherence is invisible when you evaluate feature by feature. The biggest flaw only appeared when I tried to explain the full experience to someone else. Next time I'd do a rough end-to-end walkthrough much earlier.

What's next for CodeSense

Right now CodeSense adapts within a lesson. The next step would be to adapt across a curriculum. Each lesson already produces signals: what the learner noticed, how deeply they explained it, which predictions they missed, whether they needed hints. A curriculum router could use those signals to pick the next scenario from explicit prerequisites rather than having a model freestyle a recommendation.

Topics I want to build next: debugging from logs and incomplete symptoms, database/data-model design, API reliability and concurrency, and system design tradeoffs. These could converge into a guided 0-1 project where the learner designs, builds, debugs, and evolves a system, with CodeSense challenging their assumptions along the way rather than building it for them.

Built With

• next.js
• typescript
• vercel