What it does
BoardNotes turns lecture videos into organised study notes and a textbook-style PDF. It reads the board, listens to the lecturer, and cites every point back to the exact board writing and video moment. When it can't read something, it says so. Students suggest fixes, the AI checks them, and lecturers approve. Students can also ask the lecture questions and get sourced answers.
How we built it
We designed it in Figma first: a design system plus student and lecturer prototypes. Then we built it with Next.js, FastAPI and Postgres.
The AI pipeline matches each model to its job:
- Claude Haiku scans frames for new writing.
- Whisper (fal) transcribes the audio.
- Claude Opus reads the board and writes the notes.
- Claude Sonnet checks fixes and answers questions.
A citation checker removes anything unsourced. All student and lecture text reaches the model only as data, never as instructions.
Challenges we ran into
- Moving cameras: we match frames by their text instead of their pixels.
- Partial and duplicate readings: we merge them across frames.
- An overly unsure AI: each point now scores by its best evidence, and agreement across $n$ frames boosts confidence. That cut flagged points from 6 of 11 to 1.
Accomplishments that we're proud of
Every note is traceable to its source. In our demo, the AI flagged an equation hidden by the lecturer's hand. A student suggested the fix, the AI confirmed it from another frame, and the lecturer approved it. A prompt-injection attempt was caught and flagged.
What we learned
Trust comes from citations and honest uncertainty, not a confident tone. Using cheap models for volume and strong models for accuracy keeps costs low.
What's next for BoardNotes
University sign-in and LMS integration, live processing during lectures, and practice questions generated from each lecture.
Built With
- claude
- fastapi
- nextjs
