Inspiration

Students often struggle with scattered lecture videos, textbooks, and slides. Generic AI chatbots can provide answers without connecting them to course material or considering what a student already understands. I built StudyPilot to create a source-grounded learning companion that connects course content, assessments, and personalized revision into a continuous learning workflow.

What it does

StudyPilot transforms PDFs, PowerPoint slides, textbooks, audio, and lecture videos into a structured, searchable knowledge base. It identifies topics, subtopics, concepts, and prerequisite relationships, while preserving source locations such as page numbers, slide numbers, and video timestamps. Its full visual-processing mode can also extract information from diagrams and figures using OCR and optional vision-model processing.

The source-grounded tutor retrieves relevant passages, answers questions with source citations, and flags questions that the uploaded material does not adequately cover. Optional outside knowledge is separated from source-backed answers.

StudyPilot generates custom, diagnostic, adaptive, and mock assessments containing multiple-choice, short-answer, and numerical questions. Questions are associated with topics, sources, and difficulty levels. The assessment pipeline supports answer verification, duplicate detection, cited feedback, misconception hints, and reports that identify weak topics.

Its learner model tracks topic-level mastery using Bayesian Knowledge Tracing-style updates, forgetting estimates, and learning evidence from quizzes and conversations. New students can begin with an intake and diagnostic assessment. StudyPilot then prioritizes weaker topics and prerequisites to guide subsequent practice, spaced repetition, flashcards, and study planning.

How we built it

I built StudyPilot using Python, FastAPI, JavaScript, HTML, and CSS. PyMuPDF processes PDFs, python-pptx extracts presentation content, RapidOCR handles image and scanned-page text, and faster-whisper supports local audio and video transcription. Its retrieval pipeline uses lexical search, TF-IDF, BM25-style ranking, and optional neural embeddings.

SQLite provides the default database, with PostgreSQL support available. OpenRouter connects the application to configurable AI models. Separate modules handle ingestion, knowledge-base construction, retrieval, tutoring, assessment generation, learner modelling, revision, and evaluation.

Challenges we ran into

The main challenges were preserving precise source locations across different file formats, preventing unsupported AI claims, verifying generated questions, and managing API failures and rate limits. I developed a retrieval-based grounding pipeline, source-coverage checks, answer-verification mechanisms, duplicate detection, and fallback models. When AI generation is unavailable, supported source-based practice can keep the learning workflow running.

Evaluations also exposed areas needing improvement, particularly exact citation matching, refusal of unsupported questions, and balancing reduced repetition with mastery gains.

Accomplishments that we're proud of

I integrated multimodal material ingestion, a structured course knowledge base, source-grounded tutoring, adaptive assessments, per-topic mastery tracking, a study dashboard, and personalized revision into one application.

I also implemented question and knowledge-base audits, answer-quality evaluation, and simulated-student experiments. These allow me to examine not just whether features work, but also where retrieval accuracy, grounding, and personalization need improvement.

What we learned

I learned to combine language models with retrieval, document processing, source metadata, assessment verification, and learner-state tracking. I also learned that a useful educational AI system needs more than convincing answers: it needs traceable evidence, reliable question generation, meaningful personalization, and transparent evaluation.

System evaluation and benchmarking

StudyPilot includes a built-in evaluation pipeline based on RAGAS-style metric definitions, an optional DeepEval integration, knowledge-base and question audits, and multi-session synthetic-student simulations.

The latest saved evaluation included with the project reports faithfulness of 0.589, answer relevancy of 0.400, context precision of 0.892, context recall of 1.000, and citation accuracy of 0.656 on 15 in-scope questions. It correctly declined 10 of 12 off-material questions. These are preliminary system-evaluation results, not human-validated learning outcomes or independent live DeepEval benchmark results.

In a simulation involving 30 synthetic students over six sessions, adaptive practice reduced question repetition to 24.9%, compared with 54.5% for a random baseline. However, simulated mean mastery gain was 12.5 percentage points for the adaptive policy versus 17.0 for the random baseline. This indicates that my current policy still needs improvement to balance novelty and learning gains. Simulation results do not establish real-world student outcomes.

What's next for Study-Pilot

My priorities are to improve exact citation accuracy and unsupported-query detection, refine the adaptive policy so that lower repetition also produces stronger mastery gains, run independent framework-based evaluations, and validate the system with real learners. I also aim to improve multimodal figure understanding and make personalized recommendations more reliable across longer study sessions.

Built With

• bayesian
• css
• deepeval
• fastapi
• faster-whisper
• html
• information-retrieval
• javascript
• numpy
• ocr
• openrouter
• pillow
• pyav
• pymupdf
• python
• python-pptx
• scikit-learn
• sqlite
• tf-idf