VidPop
Check what's said in a video, at the second it's said.
- Live demo: https://vidpop-ee91f.web.app
- Code: https://github.com/kaiji123/vidpop (MIT)
Inspiration
Short videos are where many people now get their news, science facts and shopping ideas, but there's no good moment to check any of it. To look something up you have to pause, leave the app, type a search and lose your place. Most of the time nobody bothers, so confident claims like "you only use 10% of your brain" go unchecked, and products people actually want are hard to find. We wanted the answer to appear inside the video, at the exact second it's needed.
What it does
VidPop adds small, dismissible popups to a video at the exact moment something worth looking up is said.
- Verify mode. When someone makes a factual claim, a card appears. Each claim is already checked in the background by Gemini with Grounding with Google Search, so the popup can show "✕ Sources disagree" or "✓ Sources agree" straight away. Tapping it opens an in-video sheet with a plain-English verdict (supported, contradicted, mixed or not enough evidence), cited sources, and Google Search suggestions. With a Fact Check Tools API key configured, published fact-checks with publisher and review dates appear alongside.
- Discover mode. When a product is mentioned, VidPop offers Google Search and price-comparison links, plus a clearly labelled sponsored card matched from a demo catalog of 120 products. Ads are matched only to what was said, never to the viewer. They never appear on claims or political content and are always kept separate from fact-checking.
- Upload your own clip. Drop in any video and VidPop transcribes it, finds the claims and products, and fact-checks them automatically, with no extra clicks.
How we built it
VidPop is a hybrid AI app with no backend. It's a static React + TypeScript site on Firebase Hosting that talks to Gemini through Firebase AI Logic.
- On-device first. Moment detection uses Firebase AI Logic's
PREFER_ON_DEVICEmode, so Gemini Nano scans the transcript locally when Chrome has it, and cloud Gemini Flash-Lite takes over when it doesn't. The UI always shows which engine ran. - Exact timing. The model returns transcript line numbers, never timestamps. Times always come from the transcript, so popups can't drift.
- Transcription. For uploads, the browser extracts only the audio track (much cheaper than sending video frames) and Gemini Flash-Lite transcribes it. We then snap each sentence start to a real speech onset detected from the audio energy, which brings popups to within about 0.1 s.
- Grounded verification in the cloud. Gemini with the Google Search tool weighs reputable sources under a neutral prompt and sees the surrounding sentences, so claims are judged in the sense the speaker meant.
- Pre-generated fact checks. Every claim is verified in the background, two at a time, as soon as a clip is scanned. Opening a claim is instant, and a claim that's still being checked never triggers a second request.
- Security. Firebase App Check with reCAPTCHA Enterprise protects the Gemini quota, and a pre-deploy script refuses to ship without it.
- It never dead-ends. Retries and model fallbacks handle overloaded models. Without any keys, a heuristic extractor and a labelled demo cache keep the experience working.
Challenges we ran into
- Deciding what counts as a claim. Our first prompt told the model to skip opinions, so "I think 2 + 2 is 5" was ignored, because "I think" made it look like one. We had to teach it that hedges don't turn a checkable fact into an opinion.
- Technically true, practically false. "Diamond is not the hardest material" was first rated supported, because lab-made materials can be harder. The speaker meant "wood is", though. Giving the fact-checker the surrounding speech and asking it to judge what a viewer would understand fixed it.
- Timing. Model timestamps are only accurate to the second, which feels wrong in a video. We solved it with transcript line indexes and audio-based alignment.
- Keeping a no-backend app safe. A Firebase web config is public, so without App Check anyone could spend our Gemini quota. Setting up reCAPTCHA Enterprise correctly, including local development, took real effort.
- Keeping ads honest. Sponsored content next to fact-checking is risky. We kept them strictly separate, labelled every ad, and banned ads on claims and political content.
Accomplishments that we're proud of
- A real hybrid split: cheap, private moment detection on-device, and grounded, cited verification in the cloud.
- Popups that land on the exact word, even in clips we've never seen before.
- Fact checks that are ready before the viewer taps, with a verdict right on the popup.
- Upload any clip and it works end to end, from transcript to moments to verdicts, with no backend.
- A neutral design: verdicts describe evidence, not people, and every answer shows its sources and the engine that produced it.
What we learned
- Prompt wording matters more than model size for classification. One word ("opinions") silently dropped real claims.
- Context changes the truth. A sentence can be literally true and still misleading, so fact-checking has to see what was said around it.
- Hybrid AI is a design choice, not just a fallback. Doing the frequent, cheap step on-device and saving cloud calls for grounded answers keeps the app fast, private and affordable.
- Security setup (App Check, API restrictions, keeping keys out of production builds) is part of the product when you have no server.
What's next for VidPop
- A browser extension and creator-platform integrations, so VidPop works on the videos people already watch.
- A shared, server-side cache of verdicts, so popular claims are checked once and served instantly to everyone.
- Gemini-based semantic ad matching and a self-serve advertiser portal with real inventory.
- Multilingual transcription and fact-checking.
- Tools for creators to see and correct flagged claims before publishing.
Built with
Gemini Nano (Chrome built-in AI) · Gemini API (Gemini 3.6 Flash, Gemini 3.5 Flash-Lite) · Grounding with Google Search · Firebase AI Logic · Firebase Hosting · Firebase App Check · reCAPTCHA Enterprise · Google Fact Check Tools API (optional) · React · TypeScript · Vite · Web Audio API · HyperFrames (demo video)
Built With
- fact-check-tools-api
- firebase
- firebase-ai-logic
- gemini
- gemini-nano
- google-search-grounding
- react
- recaptcha-enterprise
- typescript
- vite