Study Companion: A Source-Cited AI Tutor

An AI study companion that answers only from your slides, textbook and lecture video, cites the exact page or timestamp, and quizzes you to track which topics you've mastered.

Comment

Inspiration

Students study from lecture videos, slides and textbooks scattered across different places. When they ask a general chatbot for help, the answer often isn't tied to their course, can't be checked, and ignores what the student already knows. We wanted a study companion that teaches only from your own material, shows its sources, and adapts to you.

What it does

  • Unified knowledge base: upload slides (.pptx), textbook PDFs and a lecture video transcript. Every chunk keeps its origin: page, slide number or video timestamp.
  • Source-grounded tutor chat: answers carry numbered citations. Each citation opens the exact snippet, the exact PDF page, or the YouTube video at that timestamp.
  • Honest refusals: if the material doesn't cover a question, the tutor says so instead of guessing. A separate, clearly labeled button gives a general-knowledge answer, kept apart from source-backed content.
  • Adaptive quizzes: multiple-choice questions on topics the student chooses, each tagged with topic, difficulty and source location. Feedback cites the source and flags the likely misconception.
  • Learner model: a per-topic mastery estimate updates after every quiz, difficulty adapts to it, and weak topics are highlighted. A new student's first quiz doubles as a diagnostic.

How we built it

  • Ingestion: PyMuPDF (PDF), python-pptx (slides) and youtube-transcript-api (video transcripts, grouped into ~45-second windows). All text is split into chunks with location metadata.
  • Retrieval: chunks are embedded locally with sentence-transformers (all-MiniLM-L6-v2) and searched by cosine similarity.
  • Grounding: the Gemini API answers using only the retrieved, numbered sources and must cite them. Refusal has two layers: a similarity-score threshold, and the model replying NOT_COVERED when the sources are insufficient.
  • Quiz generation: Gemini writes questions from retrieved chunks. A second, independent call solves each question, and mismatches with the answer key are discarded. An embedding-similarity check rejects repeats of earlier questions.
  • Learner model: Bayesian Knowledge Tracing per topic. Let \(P(L)\) be the probability the student knows the topic, \(s\) the slip probability, \(g\) the guess probability and \(t\) the learning probability. After a correct answer:

$$P(L \mid \text{correct}) = \frac{P(L)(1-s)}{P(L)(1-s) + (1-P(L))\,g}$$

After an incorrect answer:

$$P(L \mid \text{incorrect}) = \frac{P(L)\,s}{P(L)\,s + (1-P(L))(1-g)}$$

Then a learning step:

$$P(L) \leftarrow P(L \mid \text{obs}) + \big(1 - P(L \mid \text{obs})\big)\,t$$

  • Interface: Streamlit, with Upload, Tutor Chat, and Quiz & Progress tabs.
  • Reliability: response caching, retries with model fallback, and an optional backup LLM provider, because free-tier APIs are rate-limited.

Evaluation

  • Refusal threshold: we swept the similarity threshold on a small test set and chose 0.40. Clearly unrelated queries scored far below it (cricket 0.08, cooking 0.17, geography 0.14). Queries on related topics the material doesn't cover scored close to it (convolutional networks 0.35, Bellman equation 0.38), so the second, model-based refusal layer matters for those.
  • Retrieval: on 15 in-scope questions with known slide and page locations, Hit@1 [X]%, Hit@3 [X]%, MRR [X], context precision@4 [X]%, context recall@5 [X]%.
  • Generation: [faithfulness / answer relevancy, scored by an LLM judge, or "not run because of API quota"].
  • Simulated students: 50 synthetic students over 5 sessions. Adaptive question selection removed repeats (0% vs 14% for random selection), but mean mastery gain was about the same (+0.163 vs +0.161). In this simple simulation, the benefit of adaptivity was avoiding repeated questions, not faster learning. Synthetic students validate the logic, not real learning.

Challenges

  • Keeping answers strictly source-backed while still readable.
  • Choosing a refusal threshold that rejects off-topic questions without rejecting valid ones.
  • Getting reliable answer keys, which led to the second-call verification step.
  • Free-tier API overload and quota limits (503 and 429 errors), handled with caching, retries and a fallback.

What we learned

Grounding depends mostly on retrieval quality and strict prompting, and honest testing matters as much as building features.

Limitations and what's next

This prototype covers one course unit and works on text from slides, PDFs and transcripts. It uses a flat topic list instead of a prerequisite graph, and doesn't yet extract figures or diagrams. Quizzes are multiple-choice only, mastery updates from quizzes but not chat, and faithfulness is scored by an LLM judge rather than the RAGAS library. There is no audio or Hindi support yet. Next: figure extraction, a visual topic map, flashcards for weak topics, a spaced-repetition study schedule, and multi-language support.

Credits

Lecture video: Stanford CS229 Lecture 2 (Andrew Ng, Autumn 2018). The slides and textbook chapter were self-created for this project.

Built With

  • bayesian-knowledge-tracing
  • gemini-api
  • numpy
  • pymupdf
  • python
  • python-pptx
  • pytorch
  • retrieval-augmented-generation
  • sentence-transformers
  • streamlit
  • youtube-transcript-api
Share this project:

Updates

Submission history