Rushes

Natural-language video search across your archive and the public web, powered by Gemini and ClickHouse.

Comment

Inspiration

The idea started from a genuinely boring problem: editors and archivists lose hours searching for the right shot in footage libraries with useless filenames — CLIP_0847_final_v2.mp4 tells you nothing about what's actually in it. Stock footage companies solved this for their libraries by paying humans to tag everything. Nobody's solved it for a studio's own private, unlabeled archive, because it's never been worth building an enterprise DAM system just for that.

At the same time, editors don't only search their own footage — they're constantly checking what's publicly available too. So the real idea became: what if one natural-language search covered both, clearly labeled, without ever pretending to own footage it doesn't?

What it does

You describe a shot in plain English — "a lone figure on a rain-slicked neon street" — and Rushes returns matches from your own archive (semantically indexed, fully playable) alongside relevant results from YouTube, the Internet Archive, and Pexels (metadata and a link out, never rehosted). A Gemini-powered planning agent turns your description into a dense sentence for archive vector search and a short keyword query for the public sources, runs all four in parallel, and writes a plain-English reason for every match — including noticing when a clip doesn't quite fit ("wet street, neon reflections, but has two figures" for a query asking for one).

How we built it

  • Agent orchestration: Google's Agent Development Kit (ADK), with Gemini planning each query and writing match reasons
  • Archive search: ClickHouse Cloud, queried through the official mcp-clickhouse MCP server — a real running server process over stdio, not a client library — using cosine-similarity vector search over Gemini-generated embeddings
  • Public sources: YouTube Data API, Internet Archive's advancedsearch API, and Pexels' API, called live and merged into one ranked result set
  • Frontend: Next.js, deployed on Vercel
  • Backend: FastAPI, containerized and deployed on Railway

The seeding pipeline runs every archive clip through Gemini once to generate a structured caption (title, description, tags, mood, setting), embeds the description, and inserts it into ClickHouse — the same pipeline that powers search at query time.

Challenges we ran into

Almost every hard bug here was a silent one — the kind where something looks like it's working until you check closely:

  • A packaging bug (pyproject.toml pointing at a relative README path) broke editable installs the moment the README file actually existed
  • ADK builds its own Gemini client from os.environ, separately from our app's config object — so a missing API key didn't crash anything, it just silently fell back to keyword search. Every result still came back, just worse, with no error to notice
  • The first "successful" Cloud Run-style deploy reported healthy status while ClickHouse was actually offline: a non-editable pip install changed where Python resolved the package root, which broke the MCP binary lookup and pointed our clip-serving directory into an empty folder inside the container — both failures were caught and logged as warnings rather than raised, so the health check stayed green
  • Google Cloud billing repeatedly auto-closed on new accounts through no fault of ours, which blocked Cloud Run entirely — we pivoted to Railway for the backend instead, since nothing in the hackathon rules requires the hosting platform to be Google Cloud, only that Gemini/ADK and ClickHouse are genuinely used
  • railway up's CLI upload has a hard client-side timeout that our archive's file size couldn't fit under on a slow connection — fixed by pushing to GitHub and letting Railway clone server-side instead, which has no such deadline
  • Gemini's free-tier cap (20 requests/day) meant every verification search was a real, limited resource — we had to treat testing itself as something to budget, not just the final demo

Accomplishments we're proud of

Getting to a state where we could prove the system wasn't silently degrading — writing a check that ruled out every possible fallback string shape and confirmed 0 of 20 live search results matched a fallback, rather than just assuming Gemini was doing the work because results came back at all.

What we learned

The most dangerous bugs in an agentic pipeline aren't the ones that crash — they're the ones that degrade gracefully and keep returning plausible-looking output. A health check that returns 200 is not the same as a system that's actually working end to end.

What's next

Moving archive clip storage to Google Cloud Storage instead of baking clips into the container image, and giving the seeding pipeline its own separate Gemini quota so search and ingestion don't compete for the same daily allowance.

Built With

Share this project:

Updates

Submission history