Watch Out

A Chrome extension that verifies claims made in YouTube Shorts by cross-referencing them against trusted web sources in real-time.

Comment

Watch Out

Inspiration

Short-form video is the fastest-growing source of news and "facts" for Gen Z — and the least accountable. There's no built-in way to know if what you just watched is real. We wanted to build the thing that should already exist: a fact-checking layer that lives where misinformation spreads, not on a separate website you'll never visit.

What It Does

Watch Out is a Chrome extension that adds a one-click fact-check button directly into the YouTube Shorts action bar. Click it while watching any Short, and a sidebar opens alongside the video with results in about 5 seconds.

You see:

  • An overall verification score (0–100%) shown as a color-coded bar — green for well-supported, yellow for mixed, red for likely misleading
  • Individual claim cards for each verifiable factual claim, color-coded by verdict: ✅ Supported, ❌ Refuted, or ⚠️ Unclear
  • Plain-English explanations with confidence levels and clickable source links
  • "What to check next" suggestions for unclear claims, so you can dig in yourself

No account needed. No separate app. It works right inside the Shorts feed.

How We Built It

  1. Chrome Extension (Manifest V3) — Content scripts detect the active Short via URL parsing, a MutationObserver, and polling. The sidebar renders directly into the YouTube page using vanilla JS for minimal latency.
  2. Service Worker — Acts as the bridge between the page and backend, managing a two-tier cache (in-memory + chrome.storage.local), request deduplication, and progress messaging.
  3. Next.js API Backend — Orchestrates a 5-stage pipeline: transcript extraction → claim extraction (Gemini) → evidence search (Tavily) → claim verification (Gemini) → weighted scoring.
  4. Prefetch Engine — Scrapes upcoming Short IDs from the DOM and pre-analyzes the next several videos on a dwell timer, so results are instant when the user swipes.
  5. Benchmark Suite — Automated model comparison across latency, token usage, cost, and quality heuristics with a weighted recommendation score, used to select the best Gemini model for the task.

Challenges We Ran Into

  • Transcript quality — YouTube auto-captions are full of duplicate rolling lines, timestamps, and markup. We wrote a custom VTT parser with Jaccard similarity deduplication to produce clean text.
  • Latency vs. depth — Balancing thorough verification against sub-10-second response times required priority-based timeout budgets, a sliding-window prefetch system, and a two-tier cache.
  • Model selection — Gemini models varied wildly in speed, cost, and quality. We built a benchmarking harness and tested across 5 videos for ~10 runs each to pick the right one.
  • YouTube's DOM — The Shorts action bar uses custom elements (ytd-reel-video-renderer) that load asynchronously and change structure without notice. We ran three detection strategies in parallel (navigation events, MutationObserver, polling) with setTimeout retry logic for button injection.
  • Beta frame analysis — We built a secondary pipeline that extracts claims from video frames using a vision-language model, but ran into YouTube API rate-limiting issues. The pipeline works but isn't consistent enough to ship yet.

Accomplishments We're Proud Of

  1. 5-second end-to-end latency from button click to rendered verdicts — fast enough to check a claim before the next Short autoplays
  2. Live on the Chrome Web Store — deployed and available to anyone right now
  3. Native-feeling UX — the button lives exactly where users already look on Shorts, requiring no behavior change

What We Learned

Designing inter-component contracts before writing any code was the single most valuable decision we made. We locked in the API response shape, message-passing types, and prop interfaces from the start, so each person could build and test their piece independently using mock data. Integration became a matter of plugging things together rather than debugging mismatched assumptions.

We also learned that LLM prompt design is more empirical than intuitive. The final prompts look simple, but they came from testing against dozens of real Shorts transcripts and iterating on edge cases — particularly around what counts as "verifiable" versus "opinion."

What's Next for Watch Out

  1. Frame-by-frame analysis — finish the vision pipeline that extracts claims without needing a transcript, adding a second independent fact-checking layer
  2. More platforms — bring Watch Out to Instagram Reels, TikTok, and Facebook, where misinformation spreads just as fast
  3. Creator transparency scores — aggregate verdict history by channel to surface patterns of misinformation over time

Built With

Share this project:

Updates