ThinkTrace AI

Don't just mark the wrong answer. ThinkTrace AI reads each student's written reasoning, groups the class by how it is confused, and keeps teaching until the misconception is demonstrably gone.

Comment

Inspiration

Every quiz tool in a classroom can tell a teacher that 40% of the class got question 3 wrong. None of them can tell the teacher why, and the why is the only part that changes what you do next.

We watched the same failure happen from both sides. A teacher sees a bar chart with 40% red and has no idea whether those students share one misconception or five different ones — so the reteach is aimed at an average student who does not exist. Meanwhile the student who got it wrong sees a red cross, a correct answer, and no explanation of the belief that produced their answer. They move on carrying the same broken model into the next topic, where it breaks again.

So the two halves are actually one problem. Classroom-level confusion and individual-level misconception are the same thing seen at two zoom levels, and nobody had built the loop that connects them. That is what we set out to build.

What it does

ThinkTrace AI runs one connected cycle, and the cycle does not close until the student can explain the concept in their own words without the original misconception in it.

Teacher side — LecturePulse

  • Create a live classroom, get a six-character join code, publish questions to the room in real time.
  • Every question asks for the answer and the reasoning behind it. That written sentence is the input to everything downstream.
  • The confusion map groups students by how they are thinking, not by which option they ticked. Two students who picked different options for the same underlying reason land in the same group; two who picked the same option for different reasons do not.
  • The teacher gets ranked missing prerequisites across the class, a two-minute intervention they can deliver right now, a counterexample to put on the board, and a follow-up question that separates the groups.

Student side — ConceptLens, a five-stage repair path:

  1. Diagnosis — the belief behind the answer, why that reasoning fails, a counterexample, and a confidence score. The student can confirm, reject, or flag it as unclear. Rejecting it stops the cycle, because practising against the wrong premise is worse than not practising at all.
  2. Prerequisite Detective — the shortest repair path through the concept graph: two or three short concepts, foundation first.
  3. ErrorTwin — practice built around the reasoning pattern, not the topic. A similar case, the same trap in a new domain, then a transfer question.
  4. Explain My Way / PerspectiveLab — the same idea rewritten as simple language, technical, step-by-step, low-text visual, or a real-world analogy; and in another language (Telugu, Kannada, Hindi, Spanish, French, or anything you type), optionally keeping the technical terms in English. PerspectiveLab re-explains it through people whose jobs depend on getting it right — a radiology screening lead, a warehouse robotics engineer, an assessment designer, a machine learning researcher, a child's fishing guide.
  5. Teach-back — the student explains it in their own words. Not another multiple choice. The evaluation decides whether the original misconception is actually gone.

The mastery ladder only advances on evidence: 🔴 misconception still driving the reasoning → 🟡 correct answer, uncertain reasoning → 🟢 understands and can transfer → 🔵 can explain it to someone else. Finishing the ErrorTwin set with a mistake still in it leaves you at yellow. Only a teach-back that survives the misconception check reaches blue. And whatever the student reaches flows straight back to the teacher's dashboard, so the teacher can see the loop close per student.

In our demo video the student types "i need more time to understand" — and the system refuses to mark it resolved. That is the product working, not the product failing. Nothing is marked fixed until the student can explain it.

How we built it

Next.js 15 App Router + React 19 + TypeScript (strict) + Tailwind CSS v4. Around 11,500 lines across 80 files. All AI work happens in route handlers, never in the browser.

Supabase for auth, Postgres and Realtime. Ten tables — profiles, sessions, participants, questions, responses, confusion_maps, diagnoses, practice_attempts, teach_backs, mastery — with row-level security on every one of them. Published questions and incoming responses reach the other side of the classroom over Supabase Realtime (postgres_changes), with polling as a fallback.

Claude Opus 5 with forced tool-use for structured output. Every analysis call declares a JSON schema and pins tool_choice, so the model returns a typed object rather than prose we have to regex. Five distinct analyses: the class confusion map, the individual diagnosis, the explanation rewriter, the perspective set, and the teach-back evaluator. An OpenAI adapter using json_schema is wired in behind the same interface.

A store adapter with two implementations. One Store interface, one in-memory demo implementation and one Supabase implementation. That is what lets the whole app run with zero API keys and zero configuration — clone, npm install, npm run dev, and every feature works against a deterministic analyzer and a seeded classroom. Every screen states which backend produced what you are reading, so demo output is never passed off as model output.

Challenges we ran into

Row-level security is harder than it looks, and it fails silently. Four bugs only appeared against a real Postgres, never in demo mode. INSERT ... RETURNING applies the SELECT policy to the new row, so joining a classroom failed with a policy violation while a bare insert succeeded — our first test missed it precisely because it didn't use RETURNING. A STABLE SECURITY DEFINER function cannot see rows inserted in the same statement. A partial unique index cannot serve as an ON CONFLICT arbiter, which killed ConceptLens entirely on the Supabase path while working perfectly in demo mode. We installed Postgres 16 locally with a stubbed auth schema and drove the policies directly to find them.

A join-code enumeration hole. Our first cut let any signed-in user read the sessions table — which meant reading every classroom's join code and walking into any lesson in the system. Join codes now resolve through a SECURITY DEFINER function that returns exactly one session for one exact code, and the sessions table itself is readable only by its members.

Validators that threw away good model output. We started with all-or-nothing predicates: if any field was missing, the entire response was rejected and the user saw "the model replied, but the response did not match the expected structure." Perfectly good analyses were being discarded over one absent decorative field. We rewrote every validator as a parser — take what is usable, fall back field by field — which is the right contract for LLM output.

Real-time ordering. Publishing a second question didn't show it to students. We were selecting the newest-created question instead of the newest-published one, so a draft written earlier but published later never surfaced. Caught it from a screenshot of the teacher's question list.

Refusing to fabricate. The hardest discipline was making the app say "I don't know." If a student asks about something outside the seeded lesson and no model key is configured, the demo analyzer must decline to diagnose rather than generating a plausible-sounding misconception. Confident nonsense in an education tool is worse than an empty state.

Accomplishments that we're proud of

  • The confusion map genuinely groups by reasoning. From five seeded students we get five distinct confusion types, and the grouping cuts across which option was selected. This is the thing no polling tool does.
  • The loop actually closes. Mastery flows red → yellow → green → blue on evidence and lands on the teacher's dashboard. It is not a demo stub.
  • It runs with no keys at all. The full five-stage journey works on a clean clone with no configuration, and never pretends demo output came from a model.
  • The security work is real. We verified against a live Postgres that a signed-in user who is neither teacher nor participant sees zero sessions, zero questions, zero responses and zero participants, and cannot answer as someone else.
  • Six languages, with technical terms preserved. A student can read the explanation in Telugu or Kannada while precision, recall and false positive stay in English — which is how these subjects are actually taught in multilingual classrooms.

What we learned

Structured output changes what you can build. Forcing the model into a typed schema turns "ask an LLM about this answer" into a component you can render, store, diff and act on — and it is what makes the confusion map a data structure rather than a paragraph.

Postgres RLS deserves to be tested like application logic, against a real database, with an adversarial user. Every one of our four production bugs was invisible in the in-memory path.

And parsing beats validating for model output. A predicate that returns true/false throws away information; a parser that salvages what it can is strictly better, and it is the difference between a feature that works and one that shows an error page.

What's next for ThinkTrace AI

  • Longitudinal misconception tracking — a misconception that keeps returning across topics is a different problem from one that appears once, and the data model already supports noticing it.
  • Teacher-authored concept graphs, so the Prerequisite Detective works on any syllabus rather than the seeded one.
  • Cohort view across sessions — which prerequisites this class is missing over a term, not just today.
  • Voice teach-back, so the explanation can be spoken rather than typed.
  • LMS import for existing question banks.

Built With

Share this project:

Updates

Submission history