LearnLens AI

• 

  LearnLens AI: a source-grounded, adaptive AI tutor built on your own PDFs, slides and lecture videos.

• 

  Learner Profile: level, learning goal and explanation style, plus a 3-question cold-start diagnostic.

• 

  Mastery Dashboard: overall accuracy, strongest topic, recommended focus and per-topic progress.

• 

  Evaluation: RAGAS metrics, question repetition rate and a clearly labelled synthetic personalization benchmark.

• 

  EMA mastery per topic and likely misconceptions built only from the student's own wrong answers, with source citations.

Inspiration

Course material is spread across lecture slides, PDF notes and recorded lectures. General-purpose chatbots answer from the open web instead of this course, hallucinate confidently, give no sources, and do not know what an individual student already understands. Practice questions are generic, repetitive and never get harder or easier as the student improves.

We wanted a tutor that only teaches from the student's own material, proves every claim with a citation, admits when the material does not cover a question, and adapts to the learner over time.

What it does

LearnLens AI turns a student's own PDFs, slides and lecture videos into one searchable multimodal knowledge base and builds a personalized tutor around it.

• Multimodal ingestion: PDF text and embedded figures, PowerPoint text, tables and images, and lecture videos split into timestamped topic segments. Single images can also be asked about directly.
• Source-grounded tutoring: answers come only from the uploaded material, with exact citations such as [file.pdf, Page 3], [deck.pptx, Slide 7] or [lecture.mp4, 02:10 - 03:05]. Off-material questions get an explicit refusal instead of a guess.
• Adaptive assessment: MCQ, short-answer and numerical questions. Each one is verified by a second model pass, de-duplicated, self-contained, and moves Easy ↔ Medium ↔ Hard with the student's performance.
• Learner model: per-topic mastery (exponential moving average) updated after every quiz answer, diagnostic answer and tutoring conversation, plus topic accuracy, weak topics and likely misconceptions taken from the student's own wrong answers.
• Cold start: a 3-question diagnostic sets the starting level and difficulty.
• Personalization: level, learning goal, explanation style and weak topics shape how the tutor explains, never what the material says.
• Built-in evaluation: RAGAS (faithfulness, answer relevancy, context precision, context recall), a benchmark with refusal accuracy and expected-source matching, question repetition rate, and a clearly labelled synthetic multi-session personalization simulation.

How we built it

1. Uploads are split into knowledge units, each with a fixed citation string, and embedded with gemini-embedding-2 (768-d).
2. Gemini tags units with topic, subtopic, key concepts and prerequisites in batches of 8, cached locally. Only units actually in use are tagged automatically, to save quota.
3. A question retrieves the top units by cosine similarity. gemini-3.5-flash-lite answers from only those units (plus up to 2 images per retrieved page or slide), the learner profile and weak topics, keeping citations.
4. Assessment questions cite one source unit, pass deterministic checks (valid citation, no duplicates, no references to figures the student cannot see) and a verifier pass. Their topic is the knowledge tag of the cited unit, so quizzes, mastery and the dashboard share one topic name.
5. Every answer updates the learner model: mastery = old × (1 − α) + evidence × α (quiz α = 0.30; tutoring is weak heuristic evidence, α = 0.05–0.08).
6. Learner state persists in a local JSON file; evaluation results export to JSON and CSV.

Stack: Python, Streamlit, Google Gemini API (google-genai, Files API, OpenAI-compatible endpoint for the RAGAS judge), PyMuPDF, python-pptx, NumPy, RAGAS 0.4.3.

Results

One complete Quick Benchmark run (3 cases — a small sample, not a general accuracy claim):

Raw output is in the repository under results/.

Challenges we ran into

• RAGAS with Gemini: embedding dimension mismatches broke RAGAS until we wrote a fixed 768-d Gemini embedding wrapper and used Gemini's OpenAI-compatible endpoint as the judge.
• Free-tier API quota: RAGAS makes many calls per case and hit HTTP 429. We added a quota-safe Quick Benchmark, pauses between cases, clean stopping on quota errors, and never show scores from a run with no successful case.
• Streamlit's rerun model: state written during rendering could silently overwrite later changes, and failed uploads could re-call the API on every rerun. We made these steps one-shot and retry only on request.
• Self-contained questions: models like to write "according to the figure above". We combined prompt rules, a deterministic phrase filter and a verifier that sees the same images as the generator.
• Consistent topics: free-text quiz topics fragmented the learner model, so topics now come from the cited unit's knowledge tag.

Accomplishments that we're proud of

• The full Track D loop works end to end in one app: multimodal ingestion → grounded tutoring → verified adaptive assessment → learner model → personalization → evaluation.
• Citations for PDF pages, slides and video timestamps in a single knowledge base, so one answer can cite a slide and a video moment.
• Grounding is measured, not just prompted: refusal accuracy and expected-source matching are part of the benchmark.
• Honest reporting: simulated results are labelled as simulated, and limitations are documented.

What we learned

• Grounding needs enforcement and measurement — a refusal sentence is only useful if refusals are tested.
• Cheap deterministic checks (citations, duplicates, phrase filters) save many model calls and catch errors a model verifier can miss.
• Evaluation frameworks are sensitive to provider details (embedding dimensions, rate limits); quota-aware design matters as much as model quality.

What's next for LearnLens AI

• A pilot study with real students (the current multi-session personalization result is a synthetic simulation only).
• Showing figures inside assessment questions so visual questions can be asked safely.
• Spaced-repetition scheduling from mastery estimates.
• A larger, multi-course benchmark and an independent judge model.
• Multi-user accounts and server-side storage.

Code: https://github.com/betulaltunyuva/learnlens-ai

Built With

• gemini-api
• google-genai
• numpy
• pymupdf
• python
• python-dotenv
• python-pptx
• ragas
• streamlit