Stream Forensics

Inspiration

A playback stall is easy to notice and much harder to explain. A streaming engineer needs to reconstruct what happened before the interruption, compare it with other streams, and distinguish a plausible explanation from one the telemetry actually supports.

Stream Forensics turns that investigation into an evidence-backed workflow for media delivery teams.

What it does

Ask a question such as: “Find a stream that stalled after having at least 10 seconds of video buffered. What changed in the minute before it stalled?”

The agent investigates recorded playback telemetry and produces:

A concise finding with an explicit confidence level and caveat.
Measurements linked to the database queries that produced them.
Competing hypotheses marked supported, rejected, or open.
A timeline of buffer conditions and playback events.
Inspectable query evidence and dataset context.

The interface exposes the source dataset and table information so a user can understand what the agent can—and cannot—know.

How I built it

The stack combines Gemini 3.8 Flash on Google Cloud Vertex AI, the Google Agent Development Kit, the official ClickHouse MCP server, ClickHouse, and a Next.js workbench.

The Next.js backend calls the ADK agent. The agent queries ClickHouse through the official MCP server at runtime; it does not substitute a canned narrative for database evidence. Tool access is configured read-only and the investigation has a bounded query budget.

The ingestion scripts load real public Stanford Puffer telemetry into ClickHouse. Client-buffer events describe playback state; video-sent records provide chunk delivery and transport measurements. The schema supports session- and stream-level filtering, temporal reconstruction, and aggregate comparisons.

ADK and the ClickHouse MCP server run in separate Python environments because their MCP dependency requirements differ. They communicate over stdio.

Why ClickHouse

The investigation is analytical: filter a large event table, identify affected streams, aggregate measurements, and drill into a short time window. ClickHouse stores and queries the operational evidence behind the agent’s conclusions.

The data is actual historical research telemetry, not a fabricated streaming-company incident. The app uses bounded samples of the public dataset; it does not claim to ingest the entire Puffer archive.

Challenges I ran into

The difficult part was making the reasoning auditable. A persuasive explanation is not enough if the user cannot inspect the queries or see where the evidence stops. The report connects measurements to query references and keeps unresolved hypotheses visible.

Another challenge was communicating the dataset’s scope. This is playback and delivery telemetry—not a dataset for judging television programmes or inferring audience sentiment. A correlation in these records does not establish root cause.

Accomplishments

The working prototype connects a natural-language investigation to real ClickHouse queries, a structured report, and an inspectable timeline. Dataset context and evidence are available alongside the answer.

The public repository includes source code, SQL schema, ingestion scripts, setup instructions, and an MIT license. Publication checks passed for lint, TypeScript, the production build, and connection/numeric-validation regression tests.

What I learned

An investigation tool is more useful when it can say “the evidence is inconclusive” and explain why. Preserving uncertainty and source provenance matters more than making every incident sound resolved.

What's next

Expand dataset coverage, improve investigation evaluation, and add more robust ingestion recovery and comparison workflows. This prototype analyzes historical observations; it does not monitor a live service, perform remediation, or prove causation.

Try it

Inspect the dataset information, select an example investigation or enter your own question, then compare the finding, hypotheses, timeline, and query evidence.

Source and local setup: https://github.com/varun-a-1010/stream-forensics

Data source: https://puffer.stanford.edu/data-description/

Puffer is an independent Stanford research project and does not sponsor or endorse this application.