LevelUp Live!

Inspiration

Ever zoned out in a lecture and couldn't catch up? Or lost half of it to a language barrier, a hearing difference, or a fast-talking professor? Students who are deaf or hard of hearing, still learning English, or who have ADHD or dyslexia lose the thread in lectures every day. Caption apps like Otter and Live Transcribe transcribe, but don't help you learn. Quiz apps like Kahoot need the professor to write the questions.

What it does

LevelUp Live captions a lecture live, translates it, and turns what the professor just said into a quiz question when you ask. The professor sets up nothing.

Live captions on every student's device, with the newest line largest and older lines fading
7 caption languages (including right-to-left Arabic), with English underneath
Quiz me: questions generated from what was just said, with close-call wrong answers
Upload your lecture slides as a PDF
XP grows a plant: calm motivation, not a leaderboard
Dyslexia-friendly fonts, adjustable text size, high contrast and dark mode
Sign in and join the current lecture from a link, with no app to install

How we built it

Browser mic → WebSocket → volume boost + pause detection → Whisper (on CPU) → cleanup + course-term fixing → every student's screen + MongoDB → Gemini (translation + quizzes)

Backend: FastAPI + WebSockets
Speech to text: faster-whisper, with WebRTC VAD for pause detection
Course-term fixing: RapidFuzz fuzzy matching
Database: MongoDB Atlas stores every caption live, so the transcript is always ready
AI: the Google Gemini API translates each caption into every language in the room in one call (about 1 s) and generates quizzes
Frontend: plain JavaScript with an AudioWorklet for mic capture

~ Challenges we ran into

A far-away laptop mic: we cut the word error rate from 79% to 38% with a very quiet mic, and from 22% to 13–17% at normal volume (about 7% close up).
Course terms: Whisper heard "frequent disinterpretation"; our fixer turns it into "frequentist interpretation".
Quiz quality: making wrong answers close calls on the same topic, and making sure questions don't repeat.
Delay in live recording: keeping captions close to real time while still giving Whisper enough audio to be accurate.

~ Accomplishments that we're proud of A full pipeline from the professor's voice to translated captions and quizzes on every student's screen in about a second, with big accuracy gains on an ordinary laptop mic.

~ What we learned How to use Whisper, how to process live audio, how to integrate AI tools to generate quizzes, and how to deal with delay in live recording.

~ What's next for LevelUp Live! Support for more than one live mic, a post-lecture recap and review round, and a faster model for lower lag.