Inspiration
We started from the technology, not the problem. What hooked us about ClickHouse for this hackathon was that it can hold vector embeddings and structured, relational metadata in the same table. One system for both "what does this look like" and "what scene, slate, and take is this," instead of stitching together a vector database and a separate SQL store. That felt like it was built for something specific, we just didn't know what yet.
So we went looking for a real workflow with exactly that shape of data. A friend who works in film editing gave us one: on a shoot, footage piles up fas, hours of it a da, and two things go wrong constantly. Editors waste real time scrubbing through clips looking for one specific shot, and nobody has a clear, current picture of what coverage they actually have versus what they still need to go get. Both problems are really the same problem: the metadata about what was shot exists, but it's locked inside video files nobody has time to watch again.
That's when it clicked. A clip's transcript, its shot type, its embedding, its coverage gap, all of that belongs in one row, queryable together. We weren't trying to build "AI for film" in the abstract; we were trying to prove that ClickHouse's structured-plus-vector model maps cleanly onto a tedious, real production problem.
What it does
It's built around one non-negotiable rule: it only answers from what's actually in the index. If a scene has no dialogue because it was a VFX plate that was never transcribed, it doesn't invent a line reading. On a real production, a tool that confidently hallucinates dialogue is worse than no tool at all, because someone might cut based on it.
Beyond the chat interface, it's a full six-screen production tool:
Ask — natural-language Q&A over the footage Coverage — a live grid of every scene/slate against shot size, flagging gaps straight from a ClickHouse aggregate Reels — every indexed clip, filterable, with the same semantic search built in Shot list — pickups generated automatically from the coverage gaps, not typed by hand Ingest — the pipeline itself, showing clips moving from upload through Gemini analysis to landing in ClickHouse Clip detail — the underlying evidence (timecodes, shot type, dialogue, the actual SQL) behind any answer
How we built it
Gemini watches every clip on ingest and produces the structured description, transcript, and notable elements used everywhere else in the app. ClickHouse is the system of recor, both the structured metadata (scene, slate, take, shot type, characters) and the vector embeddings used for semantic search live in the same table, so a single query can combine a keyword filter with a similarity search. An agent built on Google's ADK sits between the user's question and ClickHouse, deciding which tool to call. A coverage aggregate, a semantic search, a direct lookup, rather than the app hardcoding one query per screen. The whole thing runs on Cloud Run, hosted and live at dailies-ui-nz35hyam4a-uc.a.run.app. Every answer in the "Ask" screen ships with a "show query" affordance that reveals the actual SQL that ran.
Challenges we ran into
Getting the agent to say "I don't know" instead of filling gaps. It's easy to build a system that sounds confident; it's harder to build one that reliably refuses to answer when the underlying data doesn't support it especially for edge cases like a VFX scene with zero transcribed dialogue.
Making ClickHouse do double duty. Combining structured metadata filters with vector similarity search in a single query, instead of running two systems and merging results in application code, took some iteration to get right.
Accomplishments that we're proud of
A working end-to-end pipeline — upload → Gemini analysis → ClickHouse → queryable in the UI running on real footage, not mocked data.
An agent that visibly and correctly declines to fabricate an answer, with the evidence to back up why.
Six connected screens that all read from the same live ClickHouse data, so nothing shown is staged separately for the demo.
What we learned
The biggest surprise was how much tool-selection logic Google's ADK took off our hands. We'd assumed we'd need to hand-write the routing layer ourselves, a pile of if/else logic deciding "is this a coverage question, a semantic search, a direct lookup." Instead, we defined the tools (the coverage aggregate, the semantic search, the clip lookup) and let the agent decide which one a question actually needed.
On the ClickHouse side, the thing that stood out as we built more screens wasn't just that vectors and structured metadata could live in the same tabl. it was how naturally a single query could combine them. A question like "close-ups of the woman with the robotic arm" isn't really two separate lookups, it's one: filter on structured fields (shot type, character) and rank by embedding similarity, in one SELECT.
What's next for The Dailies Room
Scaling from six clips to a real production's worth of footage (hundreds to thousands of clips) the pipeline is already per-clip, so this is mostly a ClickHouse scaling and ingest-throughput question, not a redesign.
Demo Video
Built With
- adk
- clickhouse
- python
Log in or sign up for Devpost to join the conversation.