Scout

You promise you'll look at it later, but you won't. This will.

Comment

Inspiration

You're deep in a YouTube haul or an IG reel, you see the exact jacket you've been looking for, you tell yourself you'll come back and buy it later, but you never do.

What it does

Scout is a Chrome extension that turns any YouTube video into a shoppable list. One click on the "Save to Scout" button and we ingest the full video: title, description, transcript, top comments, and dense visual samples across the entire timeline. Each identified item comes with a clean search query so the next click is the buy. Scout is the memory layer.

How we built it

  • Manifest V3 Chrome extension with service worker, YouTube content script, popup, and a full-page saves view
  • Full-signal capture. From the watch page we pull the title,
    channel, description, and current frame via canvas.drawImage,
  • Dense timeline sampling via storyboard mosaics. Using playerStoryboardSpecRenderer.spec on every watch page, we parse the spec, pick the
    densest level that fits under a sheet cap, and feed the mosaics to
    the model as a single ordered strip. A 20-minute video becomes
    hundreds of frames covering the full timeline.
  • Gemini 2.5 Flash vision with a structured JSON response schema,
    a system prompt tuned for shoppable-product extraction.
  • Storage and UX. Products land in chrome.storage.local with
    full video context, a searchable/filterable list, JSON + CSV
    export, and per-video notes.

Challenges we ran into

  • Race conditions in the content script. YouTube's SPA fires
    yt-navigate-finish aggressively; two concurrent injections were producing duplicate save buttons. Fixed with an in-flight flag and
    a post-await re-check.
  • Embeds. youtube.com/embed rejects chrome-extension://
    origins (Error 153), and youtube-nocookie.com still fails on
    uploader embed-disabled videos. We abandoned the iframe.
  • Model hygiene. Gemini liked to fill required fields with
    "Unknown", "N/A", and "0:00" instead of null. Coercion
    layer + explicit prompt instructions got us more clean data.

What's next

  • Instagram Reels and TikTok support on the same pipeline, though all three (incl. YT) are walled gardens.
  • Direct links from each detected product to the retailer, so "one click" really is one click.
  • Price + availability refresh in the saves view.

Built With

  • captions)
  • chrome-devtools-mcp
  • chrome-extension-manifest-v3
  • chrome-storage-api
  • css
  • google-gemini-2.5-flash-(vision)
  • html
  • html5
  • javascript
  • node.js
  • openai-gpt-4o-mini
  • openrouter
  • service-workers
  • youtube-data-(storyboards
  • ytinitialplayerresponse
Share this project:

Updates