LectureAI

AI for evaluating videos

Comment

LectureAI — Hackathon Submission

Turn any YouTube lecture into a complete study environment in 60 seconds.

Inspiration

Finals week hit me hard last week. I had a pile of YouTube lectures to get through — long, dense, and easy to forget once the tab closed. Rewatching everything wasn't realistic, and skimming didn't stick. I wanted something that could turn a lecture URL into a real study environment in minutes: structured, searchable, and actually useful when you're cramming at midnight.

That's where LectureAI started. Not as a generic "AI wrapper," but as the tool I wished I had while drowning in video content.

What it does

Paste a public YouTube lecture URL — or, in Provost mode, up to 10 lecture URLs from a single course. LectureAI's multi-agent pipeline extracts transcripts, analyzes pedagogical structure, and produces something genuinely useful for real people.

Student Mode

  • Timestamped outline with jump links back into the video
  • Plain-language summaries at multiple depths (90 sec / 5 min / full)
  • Active-recall flashcards tied to specific lecture moments
  • Semantic search — ask a question, find where the lecture answers it
  • Bilingual regeneration (Spanish, French, Mandarin, Arabic, Portuguese, Hindi)
  • Fox mascot + optional voice narration during processing
  • Explore tab — a metro-style knowledge map of topics with AI chat scoped to each station

Faculty Mode

  • Private pedagogical audit across clarity, accessibility, equity, and pacing
  • Overall quality score, top priority fix, and timestamped coaching suggestions
  • Voluntary and restraint-driven — built for the instructor, not surveillance

Provost Mode

  • Ingest multiple lecture URLs from one course plus stated learning objectives
  • Produce a curriculum map comparing what was actually taught vs. what the catalog promises
  • Per-objective coverage status (Covered / Partial / Missing) with evidence and timestamps
  • Lecture-by-objective matrix, gap analysis, and recommended course-level fixes

One app. One URL. Three layered capabilities.

How we built it

Frontend: Next.js 16 (App Router), React 19, Tailwind CSS v4, Duolingo-inspired design system with fox mascot, processing mini-game, and custom components.

Backend: Express.js with a multi-agent orchestration pipeline:

Agent Job
Ingestion YouTube transcript + metadata (RapidAPI with youtube-transcript fallback)
Intelligence Topic analysis, pedagogical signals, programmatic chunking
Student Outline, summaries, flashcards
Faculty Pedagogical audit report
Provost Curriculum mapping vs. learning objectives
Search Semantic retrieval + "Explain this moment"
Chat Topic-scoped Q&A on the knowledge map

Stack: Google Gemini 2.0 Flash (analysis + generation), Gemini embeddings, Pinecone (namespace-isolated vector search), Upstash Redis (durable job state), ElevenLabs TTS, deployed on Vercel + Railway.

Processing is async — the frontend polls job status every 2 seconds with live step updates while agents work in the background.

Challenges we ran into

  • Transcript reliability — not every video behaves the same. We added RapidAPI with a fallback path and clear error handling for private videos, livestreams, and missing captions.
  • Latency vs. quality — long lectures push LLM output limits. We tuned prompts, added retry logic with exponential backoff, and kept the pipeline async so users aren't staring at a frozen screen.
  • Job persistence — in-memory storage broke on redeploys. Upstash Redis fixed that for production stability during the judging window.
  • Provost scale — processing up to 10 lectures means running the full ingestion + intelligence pipeline per video before a final cross-lecture analysis. We had to batch carefully and surface progress honestly.
  • Scope vs. polish — we chose to ship all three capabilities, but only after Student mode was rock-solid. Reliability came before feature creep.
  • The knowledge map — getting a metro-style layout that was readable and interactive took real iteration. We used v0 to nail the visual language, then wired it into our existing chat backend.

Accomplishments that we're proud of

  • A real multi-agent system — distinct agents with genuine orchestration, not three prompts in a loop
  • All three hackathon capabilities in one cohesive app: Student, Faculty, and Provost
  • Tested early with real users on diverse videos (including non-lecture content like soccer highlights)
  • Duolingo-level craft — fox mascot, 3D buttons, processing game, voice narration
  • Knowledge Map + Station Chat — a visual, interactive way to navigate and interrogate a lecture
  • Reliability hardening — Redis job store, transcript fallback, Gemini retries, structured observability logs
  • Evidence-driven Provost output — every objective claim tied to lecture moments with timestamps

What we learned

  • Judgment beats ambition — but if you can do all three well, the full stack tells a stronger institutional story
  • Reliability is a feature — especially when judges paste their own URLs live during testing
  • Students don't need more video — they need structure, recall, and fast paths back to the source
  • Visual metaphors matter — the metro map made "navigating a lecture" click instantly
  • Building for your own pain (finals week) keeps the product honest

What's next for LectureAI

  • Stronger citation UX — every AI claim visibly linked to a timestamp and transcript snippet
  • Institution-ready deployment — auth, admin controls, and integration with platforms like nebulaONE
  • Smarter flashcards — spaced repetition and difficulty tuning based on exam proximity
  • Export & offline — PDF study packs and Anki export for exam week
  • Health system use cases — training video analysis for clinical education and compliance

Built With

Share this project:

Updates