Sidecast

Inspiration

Watching a YouTube video means picking one thing: watch the video, or read the comments. Watchers rarely get to do both well, and neither helps you learn more about what you're watching. I wanted to keep the parts of YouTube that make it feel alive- real comments from real people, while adding something YouTube doesn't offer: facts about the video, checked against the live web, appearing exactly when they're relevant. The Agentic Cinema hackathon's entertainment-and-media framing, paired with Parallel's live search API, gave me a concrete way to build that idea.

What it does

Sidecast plays any YouTube video next to a synced side panel. The panel shows real YouTube comments, unranked and exactly as YouTube returns them, merged with AI-generated fact cards. Each fact card is anchored to the real transcript moment it's about, and its claim is checked against live web sources before it's shown, so it's grounded rather than asserted from a model's memory. A Fact Timeline lets you jump straight to any insight in the video without scrubbing. The panel scrolls in sync with playback automatically, holding position correctly through pausing, seeking, and speed changes.

How we built it

The backend is a FastAPI service on Google Cloud Run. A Gemini agent, built with Google's Agent Development Kit, scans the video's transcript in small batches to find checkable, non-obvious claims, then Parallel's Search API grounds each one against real web sources. Comments come from the YouTube Data API v3, transcripts from youtube-transcript-api. The frontend is vanilla JavaScript, HTML, and CSS with no build step; it uses the YouTube IFrame Player API for playback and Motion for entrance and interaction animations, and is deployed on Firebase Hosting.

Challenges we ran into

My first design ranked comments with Gemini and tagged individual claims as confirmed or disputed. It worked well on informational content but barely fired on music and entertainment videos, since there was rarely a checkable claim in a comment thread at all. I rebuilt around merged comments and insight cards instead, so grounding no longer depended on what commenters happened to write.

Placing each insight at the correct transcript timestamp was its own problem. A single Gemini call reading the whole transcript and recalling which section a topic came from was only about 20-40% accurate (naive testing) on chaotic interview content. A larger thinking budget didn't help; it just added latency. The fix was scanning the transcript in small, isolated batches so anything found was guaranteed accurate to that excerpt, then bucketing candidates by video section before picking the final set, which also fixed a recency bias that had been clustering picks in one part of the video.

Keeping the side panel visually in sync with playback also broke under seeking, since my first approach extrapolated position from elapsed time and assumed playback always moved forward continuously. I replaced it with a stateless calculation, recomputed fresh from the player's current time on every frame, which turned out to handle pausing, seeking, and speed changes correctly with no special-casing.

Accomplishments that we're proud of

• Timestamp accuracy on interview-style content went from roughly one or two correct placements out of five to about four out of five, at the same latency.
• The merged comments-and-insights feed reads as one coherent stream instead of two competing ones.
• The scroll sync holds steady through real, messy usage: seeking, pausing, and playback speed changes all work correctly.

What we learned

• Grounding quality depends heavily on content genre, and the fix was structural rather than a better prompt.
• Long-context retrieval, asking a model to recall which of many chunks something came from after reading all of them at once, is unreliable in a way that a bigger thinking budget does not fix.
• Scoping each call to a small, isolated slice of context and verifying against real content early caught problems that would have otherwise shipped.

What's next for Sidecast

1. A real Chrome extension that reuses the same panel module as a content script, so the experience can live directly on youtube.com instead of a standalone player page.
2. Using the transcript proxy to give every video real timestamp-grounded fact cards, instead of relying on a small pre-cached set.

Built With

• firebase-hosting
• gemini
• google-adk
• google-cloud-run
• google-genai
• javascript
• parallel
• parallel-search-api
• python
• vertex-ai
• youtube-data-api-v3
• youtube-transcript-api