GhostSlate

Inspiration

A stream can keep playing while its ads fail.

Viewers see a “We'll be right back” screen instead of a paid advertisement. Playback looks healthy, but advertising opportunities are being lost. Finding out why means piecing together delivery logs, ad timing, and video evidence.

I built GhostSlate to bring those clues into one investigation, helping streaming teams understand what went wrong and how much revenue is at risk.

What it does

GhostSlate helps an operator investigate missing ads, identify the affected ad supplier and device group, and understand the financial exposure.

The investigation connects three questions:

• What happened in the ad-delivery data?
• What does the sampled video frame show?
• What conclusion does the evidence support?

The operator can follow the investigation in one screen: the agent's questions, executed SQL, returned data, classified video frame, and final diagnosis. GhostSlate also proposes a targeted response for review.

When the evidence is insufficient, it declines to assign a root cause or revenue loss.

How I built it

GhostSlate combines ClickHouse with Gemini reasoning and vision on Google Cloud Vertex AI.

The agent queries ClickHouse through the official mcp-clickhouse server using the Model Context Protocol (MCP). It explores delivery records and narrows the investigation to affected groups. ClickHouse's ASOF JOIN connects delivery attempts to the relevant preceding ad-break signal.

For visual confirmation, ffmpeg extracts a frame from the synthetic clip assigned to the demo scenario. Gemini vision checks whether it shows an advertisement, regular content, or a holding screen.

The server then applies evidence requirements and calculates financial exposure from the queried records and stored ad prices. The model guides the investigation; the server owns the final diagnosis and arithmetic.

The interface uses React and TypeScript, with a Node.js and Express backend.

Challenges

The hardest part was making the diagnosis defensible.

A slow ad response alone does not prove that it caused missing ads. GhostSlate checks whether responses crossed the relevant deadline, compares affected groups with their peers, and requires visual confirmation before issuing a positive diagnosis.

Small samples presented another challenge. The system must distinguish an isolated observation from a meaningful pattern, so it requires enough distinct ad-break signals before drawing a conclusion.

Connecting these checks while keeping the investigation understandable shaped both the backend and the interface.

What I’m proud of

GhostSlate makes the evidence behind its answer visible. An operator can inspect the database results and sampled frame instead of relying on an unexplained AI conclusion.

The evaluation cases cover positive incidents, misleading signals, a negative control, and insufficient evidence. Knowing when to withhold a diagnosis is part of the product.

What I learned

A useful AI investigation needs both flexible reasoning and explicit boundaries.

The agent can decide what to investigate next, while deterministic checks keep the final business claims tied to evidence. Video adds another useful perspective: delivery records and visible content answer different parts of the problem.

What’s next

The current demo uses synthetic telemetry, seeded ad prices, and synthetic video clips. Proposed responses do not change real advertising infrastructure.

Next, I want to connect real telemetry and corresponding video samples, expand beyond the current channel and failure mechanism, and evaluate the workflow with streaming operations teams.

Built With

• clickhouse
• docker
• express.js
• ffmpeg
• gcp
• gemini
• mcp
• node.js
• python
• typescript
• vertex-ai