Lumina Live

Transform passive videos into interactive knowledge with Gemini 3-powered real-time Q&A, sentiment analysis, and creator insights

Comment

Inspiration

What it doesWhat Inspired Me

The inspiration came from two frustrating gaps in video platforms:

  1. Live Q&A Chaos: During live sessions, questions flood the chat faster than anyone can answer. As a viewer, your thoughtful question disappears into the scroll. As a host, you're overwhelmed trying to moderate, present, and respond simultaneously.

  2. The AI Integration Gap: Search engines evolved with AI—instant answers, context-aware results. Yet video platforms remain stuck as basic players. You can't ask "What did the speaker say about pricing?" and get a timestamped answer. Video platforms lack the intelligence that browsers have had for years.

Lumina Live fills this gap.

What I Learned Building Lumina Live taught me Gemini 3's true power:

Multimodal Understanding: Gemini 3 doesn't just transcribe—it understands video context, generating accurate, timestamped transcripts natively. Streaming APIs: Using generate_content_stream(), I built real-time UX where users see text appearing word-by-word as Gemini processes. Intent-Based Reasoning: Gemini 3 can detect why you're asking. "What do people think?" triggers sentiment analysis from comments, while "What is X?" pulls facts from the transcript—same input bar, intelligent routing. Structured JSON Output: Setting response_mime_type: "application/json" enabled clean analytics without fragile parsing. How I Built It Four-Layer AI Architecture Layer 1: Real-Time Transcription

Python Flask + Socket.IO backend client.files.upload() sends videos to Gemini 3 generate_content_stream() emits transcript chunks with [MM:SS] timestamps in real-time Layer 2: Intelligent Q&A

Classification prompt categorizes questions as "relevant," "off-topic," or "nonsense" Grounded extraction answers relevant questions strictly from transcript with citations Parallel threading re-checks unanswered questions as new transcript arrives Layer 3: Lumina AI Active

Bottom Bar: Intent-based routing (transcript for facts, comments for sentiment) Top Bar: Global search across all Q&A history from every uploaded video Layer 4: Creator Dashboard

JSON-structured analytics: engagement, sentiment, clarity gaps, top topics Aggregates transcript + questions + comments for holistic insights Tech Stack: Python 3.11, Flask, Socket.IO, Vanilla JS, Gemini 3 Flash Preview

Challenges I Faced Challenge 1: API Rate Limits Solution: Multi-key rotation with retry logic (3 attempts, exponential backoff)

Challenge 2: UI Blocking During Processing Solution: Moved question re-checking to background threads for smooth streaming

Challenge 3: Welcome Message Persistence Solution: Global event delegation (document.addEventListener) to hide message instantly, even after dynamic DOM updates

Challenge 4: Search Bar Confusion Solution: Clear visual separation with tooltips and result badges showing data source

Challenge 5: Slow Transcript Start Solution: Optimized polling intervals and added processing indicators for transparency

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Lumina Live

Built With

Share this project:

Updates