Inspiration
Course material arrives in fragments: multi-hour lecture recordings, dense textbook chapters, slide decks with the real content hidden in speaker notes, and pasted notes. Students burn their study time hunting for the one paragraph that explains a term and cross-checking whether a chatbot's answer is even in their syllabus. Worse, general AI answers look authoritative while being untethered from the course. I built Tutora so that a study companion can only answer from your own material, always point to the exact page, slide or timestamp, and explicitly refuse when the material does not cover the question.
What it does
Tutora is a source-grounded adaptive study companion. It ingests slides, PDF textbooks, text notes, images, audio and video with no manual preprocessing, and keeps every extracted unit tied to its origin.
Key Features:
- Multimodal knowledge base: PDF pages, PPTX slides (including speaker notes, tables and embedded pictures), text paragraphs, images and media transcripts all become citable units, each carrying its exact location.
- Exact citations, never paraphrased locations: answers link to a page number, a slide number or a media timestamp, and the Library viewer opens the original at that spot (
#page=nfor PDFs, seeking the media element for audio/video). - Grounded tutoring with real refusal: lexical BM25 retrieval plus a score threshold; when support is too weak the tutor refuses in plain language instead of answering from general knowledge, so outside knowledge is never mixed with source-backed content.
- Adaptive assessment: quizzes and mock exams in MCQ, short-answer and numerical form, scoped by the student, with every question tagged to a topic, a source location and a difficulty. Difficulty targets the mastery estimate (mastery < 0.35 → easy, < 0.7 → medium, else hard) and the weakest topics are asked first.
- Learner model and revision: per-topic Bayesian Knowledge Tracing, cited feedback and a post-assessment report with weak topics and likely misconceptions, plus a Revision page that turns the weakest topics into cited flashcards (read aloud) and an exam-date study plan with expanding review intervals.
- Cold start and honesty: a diagnostic quiz across all topics when there is no history, an explicit "no verified questions remain" response instead of recycled items, and mastery that moves only on graded answers — never on reading or chat.
How I built it
- Ingestion without preprocessing: Python pipelines using PyMuPDF for PDF text per page, visual detection and page rendering, and python-pptx for slide titles, bodies, tables, embedded pictures and speaker notes. Audio and video are transcribed through the Gemini Files API (resumable upload, poll to
ACTIVE, segment transcription, remote cleanup) with an explicit "timestamp could not be verified" warning when the model is unsure. Guards: extension allow-list, magic-byte checks, 50 MB and unit/character caps, and actionable error messages instead of silent truncation. - Retrieval and grounding: hand-written BM25 over the unit index with a light stemmer and course stopwords — deliberately lexical, so every citation is a verbatim excerpt we can verify, and so "refuse" is a stated threshold (
MIN_SCORE) rather than an opaque similarity score. Metadata (page, slide, timestamp, paragraph label) travels with every retrieved chunk. - Grounded generation: FastAPI backend that retrieves excerpts, builds context blocks and prompts Gemini in JSON response mode; the model may only write over those excerpts and must attach
[n]markers to each factual sentence. Generated quiz items are kept only if a second independent call re-solves them, reproduces the answer key and the cited excerpt exists — otherwise they are dropped. - Adaptive loop: a custom Bayesian Knowledge Tracing update (
guess 0.2, slip 0.1, learn 0.08), idempotent grading, evidence counts, and difficulty targeting derived from the mastery estimate. A tutor answer with citations offers "Quiz me on this (graded)", so a conversation can end in graded evidence that moves mastery. - Persistence: a single SQLite database (sources, units, topics, assessments, questions, answers, chat) plus the original uploaded bytes on disk so citations can reopen the real asset.
- User interface: React 18 + TypeScript (strict) + Vite 6, hand-written CSS, six pages — dashboard, library viewer, tutor, practice, course map and revision — with a configurable API base so the same build runs locally (dev proxy) and on a static host.
- Deployment: a Dockerfile on Render for the API and GitHub Pages for the static frontend (project-site asset base,
404.htmlSPA fallback,.nojekyll), with a prepared GitHub Actions Pages workflow.
Challenges I faced
- Model availability: the pinned Gemini model id I shipped started answering HTTP 404 for the project, so vision, transcription, generated questions and written tutor prose all fell back silently. We added model discovery, a
-latestalias preference, one bounded retry on 503/429 and a raw diagnostic endpoint (/api/provider/selftest) so failures are visible instead of quiet. - Hallucination prevention: strict prompt engineering was not enough on its own; grounding needed a retrieval threshold, mandatory excerpt markers, a
groundedflag, and a deterministic extractive fallback that answers verbatim or refuses. - Tagging uploaded material honestly: My first tagger matched units against every topic profile in the database and stopped creating topics after a small cap, which left a 76-page uploaded PDF with 64 untagged units and 5 units mis-tagged into machine-learning topics. We rewrote it to derive sections from the document's own structure (largest-font line per PDF page, slide titles, markdown headings) and never match one course's content into another's topics.
- Grading beyond multiple choice: scoring short answers and numerical responses reliably, and feeding only genuine evidence into the learner model, meant normalising answers, keeping submissions idempotent and stating limits rather than inflating mastery.
- Temporal and page accuracy: preserving page, slide and timestamp metadata through extraction, chunking and retrieval so the viewer can jump to the exact location the citation names.
- Shipping a SPA on a static host: running the app under a
/tutora/base path, teaching the router its base, and turning deep links into working routes on GitHub Pages.
Accomplishments that I'm proud of
- One knowledge base that unifies slides (with speaker notes), PDF pages, pasted text, images and media transcripts, with every unit traceable to its origin.
- A grounded tutor that cites the exact page or slide and demonstrably refuses off-material questions instead of guessing.
- Questions generated from the student's own uploaded material, verified before they reach the learner, and never repeated across assessments.
- A learner model with real measured behaviour: mastery moves from 0.30 → 0.69 on a correct answer and 0.30 → 0.13 on a wrong one, grading is idempotent, and weak topics drive both the assessment scope and the revision plan.
- Revision material built from the weakest topics — cited flashcards, read-aloud audio, and a study plan that respects the exam date — plus Hindi / bilingual tutor answers that keep technical terms and citation markers intact.
What I learned
- Lexical retrieval buys auditable citations and an honest refusal threshold; the price is semantic recall (a Hindi question against English material is refused, not translated), and it is worth stating that trade-off out loud.
- Pinned model ids rot. Infrastructure must discover what it can actually call, and degrade loudly rather than silently.
- A learner model is only as trustworthy as its evidence: reading and chat must not move mastery, and difficulty targeting has to be derived from the estimate rather than from the student's optimism.
- Independent verification catches plausible-but-unsupported generated questions — though mine is a same-model check, which is weaker than cross-model validation.
- An audit that only exercises the seeded sample course flatters the system; uploads are where extraction, tagging and generation really get tested.
- Citation formats have to be enforced per document type:
[n]markers for prose, page anchors for PDFs, timestamps for media, and a visible "could not verify" when a location is uncertain.
What's next for Tutora
- Intake conversation for brand-new students: a short dialogue that sets goals and an exam date, then shapes the diagnostic quiz.
- Deeper topic structure: a real subtopic tree built from chapter and section hierarchy instead of one flat level.
- Framework metrics with a judge model: executing the prepared RAGAS adapter for faithfulness, answer relevancy and context precision/recall on our team-built test set.
- Real-Time Interactive Voice Mode: spoken dialogue with Tutora while studying, with spoken citations back to the source page or timestamp.
- Persistent, multi-user storage: moving off free-tier ephemeral SQLite to a disk-backed database, with accounts so each student keeps their own mastery history.
- Collaborative Study Spaces: shared knowledge bases where classmates contribute annotated slides, unified transcripts and joint quiz challenges.
- Institutional Analytics: anonymised dashboards showing which concepts a whole cohort misses, without exposing individual scores.
Log in or sign up for Devpost to join the conversation.