Loopers

Inspiration

Students study from lecture videos, textbooks and slide decks that sit in different places. A general chatbot answers fluently, but its answers aren't tied to the course and it doesn't know what the student already understands. We wanted a study companion that answers only from the student's own material, shows exactly where each claim comes from, and adapts to what the student knows.

What it does

Loopers ingests PDFs, slide decks and lecture videos without manual preprocessing. It builds a concept graph of topics, concepts and prerequisites. Every content unit links back to its origin: a page, a slide number or a video second.

Grounded tutor chat. Answers carry citations that open the exact page, slide or timestamp. Questions the material doesn't cover are refused or flagged as outside knowledge.
Adaptive quizzes. Multiple choice, short answer and numeric items, each tagged with topic, concept, difficulty and source location. Items are verified before they are served. Students get cited feedback and a weak-topic report.
Learner model. Per-concept mastery updates after every quiz and chat turn. New students get an intake question and a short diagnostic.
Extras. A course flow map, flashcards, a cited revision brief with text to speech, speech-to-text questions, and a study schedule built from an exam date and review due dates.

How we built it

Ingestion. Whisper large-v3-turbo transcribes video, and scene detection matches transcript segments to slides. Gemini captions figures. Text, figures and video clips are embedded with google/embeddinggemma-2 into Qdrant, and unit metadata lives in SQLite.

Grounding. Retrieval runs hybrid search, then a bge-reranker-v2-m3 cross-encoder. The reranker score also acts as the refusal gate: below a calibrated threshold $\tau$, the tutor refuses or marks the answer as outside the syllabus. The language model never writes a locator. It writes markers like [E1], and the server resolves them to a page, slide or second from stored fields, so a model can't invent a citation target.

Assessment. One model writes each quiz item and a different model solves it blind. The item is served only if the solver agrees and a duplicate check passes. Generation, solving and evaluation judging use different model families, so no model grades its own work.

Learner model. Mastery is Bayesian knowledge tracing with slip $s=0.10$, guess $g=0.20$ and learn rate $t=0.05$:

$$P(L \mid \text{correct}) = \frac{P(L)(1-s)}{P(L)(1-s) + (1-P(L))\,g}, \qquad P(L \mid \text{wrong}) = \frac{P(L)\,s}{P(L)\,s + (1-P(L))(1-g)}$$

$$P(L_{n+1}) = P(L \mid \text{obs}) + \bigl(1 - P(L \mid \text{obs})\bigr)\,t$$

Stack. FastAPI backend and Next.js frontend. DeepSeek writes tutor answers and quizzes, with Groq and Gemini as fallbacks. In the deployed version the UI runs on Vercel, the API on Render, and the GPU work (embeddings, reranking, Whisper, Piper, ingestion) on a Modal T4 worker.

Challenges we ran into

Citations you can trust. Letting the model write page numbers produced wrong ones. Moving locator resolution to the server fixed that class of error, but citation accuracy is still our weakest metric.
Choosing the refusal threshold. Reranker scores sit in $[0,1]$, and thresholds taken from cosine-similarity values didn't carry over. We had to calibrate on a golden set.
Trustworthy answer keys. A learner model is only as good as its keys. Separate generator and solver models plus duplicate checks cut the errors, but repeats are still possible through one endpoint.
Running models on a small GPU. The embedding model is gated and the full stack needs a lot of memory. We moved the heavy work to a scale-to-zero Modal worker. That brought its own problems: SQLite state on a volume, a single container, and a cold start of about a minute.
Honest scoring. We chose to publish the weak numbers and the caveats next to the strong ones.

Accomplishments that we're proud of

On a demo course (OpenStax Calculus Volume 1 chapter 2, MIT OCW 18.01 Lecture 2 and a 12-slide deck), with 27 in-scope and 40 off-material questions:

Metric	Value
Retrieval hit@5 / MRR	0.93 / 0.89
Refusal honesty / over-refusal	1.00 / 0.07
Faithfulness / answer relevancy	0.75 / 0.99
Context precision / recall	0.87 / 0.81
Question repetition, adaptive vs control	0.001 vs 0.075

In simulated students (20 per profile, 10 sessions), the adaptive path gained more mastery than control (0.126 vs 0.115), and was better for 98% of students. The gap is small and depends on a simulated student model. The golden set is generated and spot-checked, and the refusal gate is calibrated on the questions it is scored on.

We also built the full path end to end: upload, ingest, cited chat, concept graph, onboarding, verified quizzes, dashboard and schedule, with an automated smoke test and a deployed stack.

What we learned

Provenance works best as data, not model output. A second model that tries to solve each question catches more bad items than asking the generator to check itself. Evaluation results are only as credible as their test set, so we wrote down where ours is weak.

What's next for Loopers

Raise citation accuracy, currently 0.57.
Add forgetting to the learner model so the schedule follows a real forgetting curve.
Send the Hindi language flag from the chat box and speak chat answers aloud.
Play the matching video second directly in the citation view.
Evaluate with LLM-played students and a hand-written test set, not only parametric simulations.