Debrief

Debrief: explain something you think you understand, and find out if it holds.
The hero. Understanding isn't a score, it's a map, solid, needs attention, or not reached yet.
Start with one of ours. Six lessons, claim by claim, from database indexes to how vaccines work.
Or bring your own. Name anything you've got down, and Debrief maps what a strong explanation needs.
The loop behind it: Explain, Probe, Reveal, Repair, grounded in the clinical teach-back method.

Inspiration

We've all had that feeling.

You're in a lesson and everything makes sense. You recognise the definition, follow the example, maybe even get the recap question right. Then later, someone asks you to explain it in your own words or use it in a slightly different situation, and suddenly... you're not so sure anymore.

You didn't necessarily misunderstand the lesson. You just never built a strong enough mental model.

We found this especially interesting with technical concepts. You can memorise what a closure is, what an API does, or what a database index is. But can you explain why it works? What changes when the situation changes? Can you reason about it without the original example in front of you?

Most quizzes are built around recognition and recall. We wanted to explore something different.

So we built Debrief around one simple question:

Can you explain it clearly, and can your explanation survive one more question?

What it does

Debrief is an AI learning tool built around explanation rather than recall.

You start by choosing a concept and explaining it in your own words. Debrief identifies the important claims behind that concept and checks your explanation against them.

Then it does something a normal quiz usually doesn't: it finds the weakest part of your understanding and asks a short follow-up question specifically about that point.

If your response reveals a contradiction or a gap, Debrief shows you the exact part of your explanation where the reasoning stopped holding and gives you a focused repair round to work through it.

If your understanding holds up, it doesn't keep asking questions just for the sake of it. You get a solid understanding result.

At the end, Debrief gives you an understanding map showing what is solid, what is unclear, what hasn't been tested yet, and what needs attention.

The idea is simple: instead of telling you that you got something wrong, Debrief tries to show you why your understanding broke.

How we built it

The core of Debrief is a stateful learning loop.

When a learner enters a concept, the system first creates a map of the essential claims that make up a strong understanding of it. The learner then explains the concept in their own words.

The AI evaluates the explanation against those claims, classifying each one as solid, unclear, needs attention, or untested.

From there, Debrief identifies the weakest claim and generates one targeted follow-up question. The learner's response is evaluated again, updating the relevant part of the understanding map.

If a genuine break point is found, the system starts a targeted repair round, usually through an application or "why" question. The final understanding state is then calculated from the individual claim states.

One of the most important decisions we made was keeping the AI's role separate from the application's logic.

The model handles concept decomposition, claim evaluation, and question generation. The application handles validation, state transitions, turn limits, and the final verdict.

This gives us a more predictable system instead of letting the model decide the entire learning flow.

Challenges we ran into

The hardest part wasn't getting an AI to generate educational questions.

The hard part was getting it to ask the right question.

A generic chatbot can ask a perfectly reasonable question, but that doesn't mean the question is connected to what the learner actually misunderstood. We needed a way to identify the important claims behind a concept, track them across multiple turns, and continuously determine which one needed attention most.

Another challenge was keeping the learner's own words as the evidence.

If Debrief says, "This is where your understanding broke," that is much more useful when it can point directly to the sentence that caused the concern. So we designed the evaluation flow around the original learner response rather than relying on a generated summary of what the learner supposedly meant.

We also had to think carefully about how the product communicates gaps in understanding.

We didn't want it to feel like the system was waiting for the learner to fail. That's why concepts like focus point, break point, repair round, and solid understanding became part of the experience.

The goal isn't to catch you out.

It's to help you find the part you haven't understood yet.

Accomplishments that we're proud of

We're proud that we turned what could have easily become another AI chatbot into an actual learning loop.

A learner can start with a broad explanation, get pushed on one specific claim, reveal a break point, and then immediately work on repairing that exact gap.

We're also proud of how central the learner's own words are to the experience.

Instead of giving you an unexplained score, Debrief can show you the sentence that exposed the gap and connect it to the part of the concept that needs attention.

That makes the feedback much more concrete and actionable.

Most importantly, we built something that feels less like taking a test and more like having your understanding genuinely examined.

What we learned

One of the biggest things we learned is that the most useful AI interaction isn't always an answer.

Sometimes it's the one question that exposes the limit of the answer you already gave.

We also learned that a single score doesn't tell a learner very much. If someone gets "72%" on a concept, what exactly are they supposed to do with that?

A useful learning system should show what was tested, what evidence supported the result, where the uncertainty is, and what to work on next.

That's why the understanding map became such an important part of Debrief.

Finally, we learned that AI becomes much more useful when it has a focused role instead of being responsible for everything. In Debrief, the model provides reasoning and language, but the application controls the workflow and decides how those outputs affect the session.

What's next for Debrief

Right now, Debrief is focused on the individual learning experience. The next step is making that experience useful over time.

We'd like learners to revisit concepts, compare their explanations before and after a repair round, and see how their understanding changes.

For instructors, we'd like to eventually surface recurring focus points across a class or cohort. If many students struggle with the same part of a concept, that's useful information for the instructor too.

We're also interested in voice-based debriefs. Explaining something out loud is different from typing it, and could make the experience feel much more natural.

There's also more work to do around calibration. A beginner learning their first programming concept shouldn't be evaluated in exactly the same way as an experienced engineer explaining an advanced topic.

But the bigger idea stays the same:

We want to help people find out whether they actually understand something, instead of just whether it feels familiar.