-
-
A cited answer with the source diagram shown under it (notes page 5).
-
A cited answer from the lecture video, with the timestamp in the citation.
-
A question outside the material is refused instead of guessed.
-
Adaptive quiz: question, mastery score, and difficulty chosen by a fixed rule.
Why I built this
When I was studying, I ran into these problems myself: my material was spread across lecture videos, PDF notes and diagrams, and when I asked an AI for help I couldn't tell where its answer came from or whether it was even in my course. I built this to solve that problem for other students, and also to learn by building it. I wanted a tutor that shows its source every time, and says "I can't answer this" when the material doesn't cover the question.
What it does
You give it PDFs and lecture videos. It reads the text, transcribes the video, and pulls the diagrams out of the PDFs. Then you can do two things with it.
Ask questions. The answer is written only from the chunks it retrieved, and it comes with a citation like notes.pdf, page 5 (diagram) or video.mp4, 00:58. I build that citation from the metadata I saved while ingesting, not from what the model says, so the model can't make up a page number. Every citation is also a link: clicking it opens the PDF at that page or the video at that second. If a diagram helped answer the question, it appears right under the answer.
Take a quiz. Questions can be multiple choice, short answer or numerical, and each one carries a difficulty and a source location. A new student starts with a short diagnostic: easy, medium, then hard, and after that the difficulty adapts. The language model writes and grades the questions, but it does not pick the difficulty. A fixed rule does that, based on recent answers, so the behaviour is easy to explain and repeat. Each topic gets a mastery score. After every answer the student sees the expected answer with its source, and at the end there is a short report with accuracy and the likely misconception.
To keep the quiz honest, a second model cross-checks every answer key, and a question that is too similar to an earlier one is regenerated.
If a question isn't covered by the material, it refuses. Two checks decide that: how close the best match is, and whether the model itself says the retrieved text is enough.
How I built it
PDF text is split into chunks that keep their page numbers. Videos go through ffmpeg to get the audio, then Whisper transcribes it and every chunk keeps its timestamp. PyMuPDF pulls the images out of the PDF, and a vision model writes a short description of each one. Those descriptions are stored like any other chunk, which is how a diagram can be found by a question. Every chunk also gets a topic, subtopic and key concepts. Everything sits in ChromaDB and the app is Flask. The models all run on Groq: openai/gpt-oss-120b for answers, tagging and quizzes, whisper-large-v3 for transcription, and qwen/qwen3.8-27b for the diagram descriptions and the answer-key check.
What went wrong along the way
- RAGAS wouldn't import in my environment because of a broken dependency upstream. I wrote the four metrics myself (faithfulness, relevancy, context precision, context recall) with a language model as the judge. So my numbers are not RAGAS scores, and I don't want them read that way.
- Once I added the diagram chunks, my refusal distance threshold stopped being reliable. Questions like "What is the capital of India?" and "Explain photosynthesis" landed below it. The model's own sufficiency check still refused them, but the threshold alone would have let them through.
- The first quiz questions came out in Hindi, because the lecture is in Hindi and the prompt never said otherwise. One line in the prompt fixed it.
- Long evaluation runs hit the free-tier rate limit on Groq, so I had to add retries.
- The vision model is a preview model, and I haven't manually checked every diagram description it produced.
What I learned
A distance threshold tuned on two documents doesn't survive new kinds of content. Building citations from my own metadata, instead of trusting the model, removed a whole class of made-up sources. And a small test set tells you very little, which is why every number below comes with its size.
Results
These come from small samples and should be read that way.
- Question answering (10 questions from the material, 6 from outside it): faithfulness 0.90, answer relevancy 0.80, context precision 0.45, context recall 0.97. All 6 outside questions were refused. The right source file was retrieved for all 10 questions, but only 4 of them have a gold page to check against, so the page-level evidence is thinner. The reference answers were drafted with AI help and not exhaustively checked against the source.
- Context precision of 0.45 means fewer than half of the retrieved chunks were relevant. Retrieval finds the right material, but it also returns more than it needs.
- Simulated learners (3 sessions of 8 questions each, one run per profile): mastery gain was +39 for the strong profile, +19 for average and +2 for weak, with no repeated questions. Correctness in this simulation is sampled from a hand-picked table, so it shows that the adaptive logic responds, not that real students learn this way. Each profile was run only once.
What's missing and what's next
The biggest gap is that I did not build prerequisite links between concepts. I tag topics and concepts, but I don't say which concept depends on which. There is also no upload page: ingestion runs from the command line. Mastery is a simple score, not a proper learner model like Bayesian Knowledge Tracing. Next I would want a bigger test set, a repeat check that understands meaning and not just word overlap, and support for diagrams drawn as vector graphics, which my extraction misses.
AI tools used: Claude (Anthropic) for coding assistance. The README lists which parts were AI-assisted.
Log in or sign up for Devpost to join the conversation.