Recap

• 

  Video avatar

• 

  Chat interface

• 

  Landing page

Inspiration

I built Recap because of a personal academic disaster.

After catching pox, I missed an entire semester of classes and coursework. I recovered only two weeks before final exams and had to learn in fourteen days what everyone else learned in four months. But lecturers intentionally keep their slides minimal so that students must attend class, which meant I had almost no material to learn from. For one course, I was lucky enough to find recorded lectures—and that ended up being the only course I scored an A in.

That experience made the problem painfully clear: when students miss class due to circumstances beyond their control, they lose access to the teacher’s explanations, examples, and reasoning. Slides alone are not enough.

So I reached out to my lecturers, and they agreed to record their lessons and provide their course materials. Using these, I built Recap: a platform that hosts the lecture videos and creates digital lecturer twins—AI avatars trained on each instructor’s own content. Students can chat with these avatars anytime to get accurate explanations, ask questions, revisit missed concepts, and receive academic support 24/7.

What it does

Recap hosts lecture videos and creates digital lecturer twins—AI avatars trained on each instructor’s own content.

Students can:

• Chat with the avatars anytime
  
• Get accurate explanations
  
• Ask follow-up questions
  
• Revisit missed concepts
  
• Receive academic support 24/7

How we built it

We collected each lecturer’s course materials—slides, PDFs, images, math, and handwritten notes—and built a preprocessing pipeline to standardize them.

The system:

• Converts mixed-format content into structured text
  
• Uses OCR for image-based slides
  
• Preserves LaTeX math
  
• Feeds everything into a multimodal LLM for grounding
  

The lecturer avatars are powered by:

• A custom-trained model aligned to the lecturer’s style
  
• LiveKit for real-time voice and video interaction
  
• Cloned voice and facial animations to make the avatar feel natural and responsive

Challenges we ran into

• Messy raw materials: slides, PDFs, images, math formulas, and inconsistent formatting
  
• Building a preprocessing pipeline that could standardize all content without losing meaning
  
• Making the multimodal LLM reason correctly across text, images, and math
  
• Ensuring the avatars’ responses reflect the lecturer’s actual teaching style
  
• Cloning the lecturer’s voice and creating face-synced animations
  
• Maintaining low latency during real-time conversations

Accomplishments that we're proud of

• Fully functional pipeline that ingests and understands real course materials
  
• Deployed digital lecturer avatars that feel like the real instructors and answer accurately
  
• Achieved real-time voice interaction using LiveKit + LLM grounding
  
• Turned a personal academic problem into a working, scalable technical solution

What we learned

• Designing multimodal preprocessing pipelines for messy educational content
  
• Aligning large language models to a specific instructor’s teaching materials
  
• Integrating real-time avatars with video, audio, and LLM reasoning
  
• The challenges of building a scalable, accurate, interactive educational system

What's next for Recap

• Expand to more lecturers and departments
  
• Add analytics dashboards for instructors to track student questions and struggles
  
• Incorporate automatic quiz and exercise generation from lecture content
  
• Implement a timeline-based recap mode that summarizes entire missed weeks in minutes
  
• Build mobile apps so students can access their lecturer twins anywhere, anytime
  

Recap transforms every lecturer into a 24/7 source of accurate, accessible academic support—solving a real problem students face every year.

Built With

• anam
• assemblyai
• elevenlabs
• fastapi
• groq
• katex
• langgraph
• livekit
• opencv
• pinecone
• react
• shadcn/ui
• tailwind