Progress section
Assessment section
Notes
Flashcards section
Summary section
Citations

Inspiration

Students rarely learn from a single source. Their study material is scattered across PDFs, presentations, handwritten notes, diagrams and recorded lectures. Finding an explanation is only part of the challenge—students also need to understand where it came from and what to practise next.

We built Aura to bring these materials into one learning workspace: an AI tutor that helps students understand their own content, revisit its sources and practise according to their progress.

What Aura does

Aura accepts documents, presentations, JPEG/PNG images, audio and MP4 videos. Students can ask questions, generate study notes and flashcards, and take assessments based on selected material.

Its main features include:

Source-linked answers with navigation to document pages, presentation slides, images or media timestamps.
Visual interpretation of diagrams, screenshots and sampled video frames.
Adaptive practice informed by topic-level mastery and assessment history.
Numerical assessments with deterministic calculation and unit checks.
Misconception feedback that presents possible misunderstandings rather than treating them as certain diagnoses.
Separate labels for additional background, visual interpretations and quoted source excerpts.

How we built it

We built the frontend with React, TypeScript and Vite, and the backend with Python and FastAPI. PostgreSQL with pgvector stores accounts, learning progress, extracted passages and embeddings.

Aura uses retrieval-augmented generation: it extracts content, creates searchable embeddings, retrieves relevant passages and generates answers with citations.

Our model stack includes:

Qwen3-Embedding 0.6B for semantic retrieval.
Qwen2.5 3B through Ollama for text generation and verification.
Gemma 3 4B through Ollama for visual interpretation.
Whisper through faster-whisper for speech transcription.
RapidOCR for recognizing text in images and scanned material.

We use Bayesian Knowledge Tracing to estimate topic mastery from assessment results. Local model execution supports CPU operation, with GPU configuration available for compatible systems.

Challenges we faced

The hardest challenge was making different file formats provide useful evidence. A successful upload did not always mean successful extraction. Documents could lose table relationships, videos could contain no usable speech, and OCR could misread small labels.

We improved document extraction to preserve surrounding context, added visual sampling across videos, and routed image questions to the original image. We also corrected cases where internal extraction headers appeared as answers.

Reliability created another tradeoff. Strict verification sometimes rejected useful explanations, while relaxing it risked unsupported claims. We kept source checks and introduced clearly labelled excerpts when a full explanation could not be verified.

Performance was also challenging. Running transcription, embeddings and multiple vision checks on a laptop can take minutes. We added background preparation, batching, model reuse and setup diagnostics, while retaining CPU support.

What we learned

We learned that building an AI tutor requires careful coordination between extraction, retrieval, generation, verification and interface design. A stronger model cannot compensate for missing or corrupted evidence.

We also learned to distinguish implementation checks from real learning outcomes. Controlled tests and synthetic student simulations help evaluate system behaviour, but they do not establish factual accuracy on every upload or demonstrate improvements in human learning.

What’s next

Our next steps are to benchmark Aura on real course materials, improve transcription and visual coverage, reduce preparation time, and evaluate the adaptive learner model with students.

We want Aura to make learning more accessible while keeping the connection between an explanation and its evidence visible.