Misconception Map

An AI tutor that turns students' wrong answers into a living map of misconceptions, then teaches straight from it.

Comment

Why I built this

I run a one-person AI operation: agents that research, write, and publish for me every day. I've been building tory.my as a personal project, an AI companion built on a personalized knowledge graph. It's sitting in App Store review as I write this. While building its language-learning feature I kept recording learners' progress and failures, and it left me with a question I couldn't shake. A log of mistakes isn't learning. What would it take to actually learn from the mistakes?

Good human tutors already know the answer: they pay attention to how a student is wrong. A wrong answer is the densest signal a learner produces, and today's tools throw it away the moment they mark it incorrect.

Misconception Map is that question, answered for math. The knowledge-graph bones aren't a coincidence either. Modeling a person as a graph is what I do in tory.my; here, the graph models what a student misbelieves.

What it does

Students do a short daily session. Questions are generated live by GPT-5.6 for whatever concept the student is weakest in, and here's the important part: the wrong choices aren't random. Each distractor is engineered around a documented misconception, like "multiplication always makes numbers bigger" or "add numerators and denominators straight across". Pick one, and the system knows which belief produced the error. That belief becomes a node in the student's knowledge graph.

The map page shows the graph: teal for concepts, red for misconceptions, sized by how much evidence has accumulated. Clicking a red node shows the actual answers that put it there. The next day's micro-lesson is generated against the strongest red node, not against a curriculum page.

The teacher view is the part I'd actually sell: "3 of 8 students share the same misconception at 78% strength," ranked by students affected times average strength. A gradebook can tell you who failed. It can't tell you that a third of the class shares the same broken mental model.

One more thing: switch the app to Korean or Japanese and GPT-5.6 writes the questions, diagnoses, and lessons natively in that language. It's generation, not translation, so the math word problems actually read like they were written for that classroom.

How the three GPT-5.6 models split the work

I ended up using all three variants, routed by what each is good at. The routing came from real measurements, not vibes.

  • sol (xhigh reasoning) writes questions and lessons. It's the best crafter and the slowest: I measured 9s, 28s, even 71s per question. So it only runs where latency is invisible, prefetching the next question while the student reads feedback, and generating lessons.
  • terra (xhigh) handles live diagnosis of free-text answers, because that's where a student is actively waiting and terra sits at the depth/latency sweet spot.
  • luna does taxonomy matching. Classification is shallow work; luna runs in parallel with terra and adds zero wait.

When a student picks a pre-tagged distractor, no model runs at all; the match is deterministic and instant. The graph update itself is plain unit-tested TypeScript. If every LLM call failed tomorrow, the app would degrade to recorded fixtures instead of erroring. That ladder exists because a judge should never see a 500.

Built with Codex, including the debugging

The whole codebase came out of one Codex session: schema, pipeline, D3 map, i18n, UI passes. 34 commits, each one gated by npm run build && npm test before it was allowed to land. The /feedback thread ID is in this submission.

The loop that surprised me: when live calls started dying with ERR_STREAM_PREMATURE_CLOSE, I fed the raw measurements back into the session and Codex diagnosed its own SDK's fetch shim as the culprit, then migrated the integration to native fetch. Korean generation later corrupted its own JSON output (reasoning text leaking into a slug field, a fun bug), and the same measure, feed back, fix, gate loop closed it. I never opened the SDK source myself.

Honest limitations

The misconception taxonomy covers one domain (middle-school fractions, 15 concepts) because building a good taxonomy is the real work. It's a product asset, not prompt garnish. Multi-subject taxonomies, spaced repetition off graph decay, and LMS integration are the obvious next steps. Longer term I think the teacher dashboard is a seat-based subscription; schools don't have this instrument today.

Built With

Share this project:

Updates