Walkout

Every platform knows when viewers stop watching. None know why. Walkout reads 13M playback events in ClickHouse, then has Gemini watch the film at that exact timecode and name the cause.

Comment

Inspiration

Every streaming platform can draw you a retention curve. At three minutes forty, the line falls off a cliff and thousands of people leave. Every platform can show you that. Not one can tell you why.

That gap is expensive. A scene that drags and a CDN that stutters produce an identical cliff on the chart, and they demand opposite fixes. One is a note for the editor. The other is a ticket for the app team. Guess wrong and you recut a film that was never the problem.

Walkout was built to close that gap. It finds the moments people leave, then it goes and watches the film at that exact second to find out what was on screen when they left.

What it does

Walkout is a scene-level abandonment-forensics agent. Give it a title and it runs a four-step investigation.

  1. Where they left. It runs survival analysis over 13 million playback events in ClickHouse. Per-second hazard rate against a robust post-warmup baseline, binomial z-test, completions censored, credits excluded. It returns the three real cliffs, to the second.
  2. Who left. It pulls the cohort breakdown for each cliff: device, platform, app version, subtitle language, rebuffer rate. If the exits are flat across every cohort, it is the film. If they concentrate in one cohort, it is delivery.
  3. What was on screen. It hands that exact timecode to Gemini, which watches the actual footage in that window and describes the scene, the pacing, the audio, the subtitle state.
  4. What to fix. It writes a verdict per cliff with a cause, a confidence, and a named owner. Not "retention dropped." Something a person can act on Monday morning.

On our demo title it finds three cliffs and returns three different causes:

  • 03:40 - Pacing. 3.1x the baseline hazard, 6,602 excess exits. Flat across every cohort, playback perfect. Gemini watches the window and finds a long dialogue-free stretch. Owner: the editor.
  • 02:10 - Localization. A subtitle gap concentrated 2.1x in one language track. The film is fine. Owner: the localization manager.
  • 09:20 - Technical. Android app 4.2.1, rebuffering at 13.01% against a 1.51% baseline. The agent's own verdict says "Do not recommend an edit to the film." Owner: the app engineer.

Three cliffs. Three causes. Three different people to call. That is the whole product.

How we built it

The agent is Google's Agent Development Kit driving Gemini. Five read-only tools, roughly seven tool calls per investigation, about 110 seconds end to end, streamed to the browser over SSE so you can watch it think.

The warehouse is ClickHouse Cloud holding 13,065,665 playback events across 250,000 sessions. The agent does not talk to it through a hand-rolled client. It talks through the official mcp-clickhouse MCP server over stdio, wired into ADK's own MCPToolset. Every number on screen came out of a live query during the run.

The vision step is Gemini reading the video file directly at the disputed timecode, not a summary, not a transcript. Windows are capped at 60 seconds and cached per window.

The stack: Python, FastAPI, ADK, Gemini, ClickHouse Cloud via MCP, Docker, Railway. The UI is hand-written HTML and CSS on a design-token system, no framework.

The analysis is deliberately boring statistics, because the agent's judgment should sit on top of numbers it did not invent. Hazard rate per second, a robust baseline that ignores the warmup period, a binomial z-test for significance, and a cohort concentration ratio that acts as the story-versus-delivery discriminator.

Challenges we ran into

A dependency conflict with no clean resolution. ADK pins mcp>=1.24,<2. The official ClickHouse MCP server needs 2.x. There is no version of Python that satisfies both. The fix was to stop trying: the MCP server lives in its own virtualenv and the two processes talk over stdio, which is exactly the boundary MCP was designed for. The Dockerfile reproduces that split at /opt/mcp.

Telling a story cliff from a delivery cliff. This is the whole product, and early versions got it wrong because a cohort with few sessions produces wild ratios. The answer was a concentration floor and a rebuffer lift floor, so a cliff is only blamed on delivery when the evidence is both large and concentrated.

Free-tier quota as a design constraint. Twenty Gemini requests per day per model, and a full run costs about seven. So the app treats a spent quota as a first-class state, not a crash. There is one error classifier for every failure that can reach a browser, and when the quota is gone the page says so plainly, explains the arithmetic, and points out that the ClickHouse queries and the stored report still work.

Making a statistics page feel like a product. Hazard rates and z-scores are not naturally exciting. The site had to make a cliff feel like something that happened to real people, which meant showing watch-hours lost rather than percentages, and putting the verdict next to the footage instead of next to the math.

Accomplishments that we're proud of

Nothing on the site is a mock. Every cliff, every cohort ratio, every scene description came out of a live ClickHouse query and a live Gemini call. The README points at the exact file and line where each required integration runs.

The agent disagrees with itself in a useful way. On the Android cliff it explicitly refuses to recommend an edit. An agent that knows when the answer is "this is not a film problem" is more valuable than one that always has a note.

It also ships: public repo, Apache-2.0, a hosted URL, an OpenAPI surface with docs, 75 passing tests and an eval suite that runs green.

What we learned

The hard part of an analytics agent is not the model, it is giving the model a small number of trustworthy numbers and a clear question. Once the hazard analysis and the cohort breakdown were solid, the reasoning quality jumped, because the agent stopped guessing and started arbitrating between two well-posed hypotheses.

We also learned that MCP earns its keep at exactly the moment two ecosystems refuse to share a process. The version conflict that looked like a blocker turned out to be the argument for the architecture.

What's next for Walkout

Multi-title comparison, so a studio can ask whether a cliff is this film's problem or a pattern across a catalogue. Writing verdicts back into ClickHouse as a first-class table so retention decisions become queryable history. And an alerting mode: watch a new release for its first 48 hours and open the investigation automatically the moment a cliff appears, while there is still time to fix it.

Built With

Share this project:

Updates

Submission history