Remembrall

Remembrall watches your lecture and quizzes you every 10 minutes, auto-pausing the video to test retention and resuming when you're done. Active recall, built into your browser.

Comment

The Problem

Passive lecture watching has terrible retention. Students zone out, videos play in the background, and nobody re-watches. Active recall at spaced intervals is the most evidence-backed learning technique in cognitive science, but nothing enforces it automatically during a lecture.

What Remembrall Does

Remembrall is a Chrome extension that watches your lecture screen in real time using VisionAgents' Gemini VLM. Every 10 minutes of actual playback, it auto-pauses the video and overlays a 3-question multiple choice quiz based on what was just covered. Answer, review your results, and resume, all without leaving the tab.

Unlike the Remembrall in Harry Potter, ours actually tells you what you forgot.

How We Built It

  • The Chrome extension captures the lecture tab via getDisplayMedia and streams frames at 1fps over a WebSocket to a local FastAPI backend
  • VisionAgents processes the incoming frames using Gemini VLM, sampling 10 frames per checkpoint for efficiency
  • Every 120 frames of active playback, Gemini generates 3 contextually accurate MCQs based on what was on screen
  • The quiz is pushed back to the extension over WebSocket, the video auto-pauses, and the overlay renders
  • On completion, the video auto-resumes

Challenges

  • VisionAgents' simple_response() was ignoring our sampled frame list and sending the full buffer to Gemini, causing near 2-minute response times. Fixing this required clearing the VLM buffer and reloading only sampled frames before each API call.
  • Making the checkpoint timer track actual playback time rather than wall clock time, so pausing the lecture pauses the quiz timer too.
  • Running quiz generation as a non-blocking async task so frame ingestion continues uninterrupted while Gemini processes.

What's Next

  • Audio transcription via Deepgram or Whisper for richer context beyond screen content
  • Playback speed awareness so 2x speed students get appropriately timed checkpoints
  • Spaced repetition across sessions to resurface missed concepts
  • Voice Q&A using VisionAgents' full voice pipeline
  • LMS integrations for Canvas, Panopto, and Coursera

Built With

  • chrome-extension-mv3
  • fastapi
  • gemini-vlm
  • ngrok
  • python
  • uv
  • visionagents
  • websocket
Share this project:

Updates