Socratic Code Tutor

Debug it yourself — we'll only ask questions. GPT-5.6 tutors students through real in-browser Python runs with escalating hints, never the fix.

Comment

Inspiration

Office hours and TA time don't scale with class size, so most "why is my code broken" questions get answered by whatever's fastest — a friend, a forum, a general chatbot — and the fastest answer is almost always a corrected snippet. That's precisely the moment where the real learning happens: forming and correcting a mental model of what the program is actually doing. Answer-first help skips it, and the same misconception shows up again on the next assignment. We wanted a tutor that could see what the code actually does at runtime and was structurally incapable of just fixing it for you.

What it does

A student pastes or picks a buggy Python sample, runs it against a real Pyodide interpreter in the browser, and gets a structured traceback plus a step-through execution trace of real variable state. GPT-5.6 tutors them through a visible four-rung hint ladder — conceptual question, localization, mechanism, near-solution scaffold — that never reaches "here's the fixed code." Every model reply is screened server-side, after generation and before delivery, so the no-solution rule holds even if a reply drifts. Each session gets tagged with a misconception category, and a teacher view aggregates those tags across a class with export to JSON/Markdown. No login, no database — everything runs from the browser and Vercel.

How we built it

The contract, UX spec, and trust-zone plan were written before any code. OpenAI Codex, driving GPT-5.6, then implemented the core application from a work-package playbook in one continuous primary session covering the sandboxed Pyodide runner, the prompt builder, the server-buffered progressive-reveal guardrail pipeline, the hint ladder, rate/cost caps, the trace visualizer, and the teacher-report/export path. GPT-5.6 does double duty in the finished product: it's the model that generates every tutor turn and misconception tag at runtime, streamed via the OpenAI SDK directly (not an abstraction layer, so the guardrail can sit between the model and the client). Student code never leaves the browser; it executes only inside a Pyodide Web Worker. Model output is buffered server-side and run through a deterministic screen at each sentence or code-fence boundary — a flagged chunk halts the flush and substitutes a safe fallback question, so nothing unscreened is ever shown.

Challenges we ran into

The guardrail is the hardest part of this build precisely because "never show a fix" has to hold against adversarial phrasing, not just the obvious case. A consolidated four-part review (TypeScript correctness, React/streaming lifecycle, a security/STRIDE pass, and manual solution-leak verification) found real bypasses — inline single-backtick code spans and gerund-phrased instructions ("try changing line 4 to...") were slipping past rung-2 and rung-4 checks, and per-chunk screening couldn't catch a fix split across streaming boundaries. Codex fixed the detector, made screening cumulative across the whole turn instead of per-fragment, and added integration tests that drive the real streaming pipeline — all in one follow-up pass, with the test count strictly increasing.

Accomplishments that we're proud of

A guardrail that's been adversarially tested against solution-shaped and injection-style prompts with zero runnable fixes reaching the client; an execution trace that reflects real Pyodide sys.settrace output rather than a simulated one; and a teacher view that turns individual sessions into an aggregate, actionable pattern — all with no backend database, live in production, with 84 passing tests and a real-runtime e2e suite.

What we learned

Streaming and "never reveal too early" are in direct tension — you cannot un-show a token. Solving that meant giving up naive first-token latency in exchange for a hard "nothing unscreened reaches the client" invariant, and testing that invariant against the actual streaming boundaries, not just a full-string check after the fact.

What's next for Socratic Code Tutor

JavaScript execution alongside Python, a model-graded second guardrail layer for defense in depth, and multi-device teacher aggregation so a teacher sees the whole class across machines — beyond the current single-browser scope.

Built With

Share this project:

Updates