Dailies

A camera assistant wears smart glasses through a shoot and walks off set with the continuity paperwork already written.

Comment

Inspiration

A script supervisor stands just off set with a clipboard. Between every take they write down where the mug sat, whether the jacket was buttoned, which hand held the phone. It is the last job on a film shoot still done entirely on paper, and the failure mode is quiet: when a line gets missed, nobody finds out until the edit. By then it is not an edit problem, it is a pickup day. A published 2026 survey of forty US production markets prices a fixed six-person, ten-hour shoot day at $1,440 in the cheapest market (Santa Fe, New Mexico) and $3,020 in the most expensive (Boston, Massachusetts).

https://giggster.com/guide/reports/film-shoot-costs-major-us-cities-2026/

I did not set out to build a film tool. I bought Ray-Ban Meta glasses for an unrelated reason and spent a while proving what could actually be got out of them, including a soak test that held a stream for 25 minutes and 45,002 frames when the stock camera app caps a clip at three. Once I knew the camera was genuinely reachable from my own code, the question became which job is done by a person whose eyes are already pointed at the right thing and whose hands are full. Continuity is that job.

What it does

The camera assistant wears the glasses through the shoot. Gemini reads every take and extracts what is actually in frame: each prop, where it sits, what state it is in, a jacket buttoned or open, a glass full or empty, a phone in the left hand or the right. That becomes structured rows in ClickHouse.

At wrap the crew asks a question in plain English, "which takes is the smartphone missing from?", and a Gemini agent writes its own SQL, runs it through the official ClickHouse MCP server against a ClickHouse Cloud cluster, and answers, showing every statement it ran.

The rolling view is the same instrument in its operating mode. Pick a reference take, press Roll, and every few seconds the current camera frame is checked against what that reference recorded. When something contradicts it, the verdict is spoken into the wearer's ear through the glasses' own speakers, because a script supervisor is watching the scene, not a screen. Finding the mug on the wrong side at wrap documents a reshoot. Finding it twenty seconds into the take prevents one.

Median verdict is 4.4 seconds on the deployed service. Every check is written back to ClickHouse with the take it was checking, and so is every other model call the system makes, as a row in agent_runs carrying its model, latency, outcome and any fallback. So the agent can be asked about its own behaviour through the same box it answers shoot questions in. There is no second dashboard.

It also runs from any webcam in a browser, so this can be tested without owning the hardware.

How we built it

Capture is a Kotlin and Jetpack Compose app on the Meta Wearables Device Access Toolkit, which is what makes the wearer's point of view reachable from my own code rather than only from Meta's app.

Everything else is a FastAPI service on Cloud Run. Gemini runs through Vertex AI with google-genai and Application Default Credentials: 3.6 Flash for extraction, reconciliation and the question agent, 3.5 Flash for the rolling check because it is faster and the recall test said it could be trusted there.

ClickHouse Cloud is behind every read path, not one endpoint. Takes and observations are a columnar append-only fact table, because comparing one object across takes is the core access pattern and a star schema would buy nothing but joins. The question agent talks to it through the official mcp-clickhouse server, driven by an explicit Gemini tool-calling loop, so the model is the caller and the server is the authority.

The one number I refuse to produce is a count of reshoots prevented, because my code would be both causing and counting it. The two numbers I do quote are metered by someone else: Google prices the tokens and the industry prices the shoot day.

Challenges we ran into

A head-mounted camera breaks assumptions a tripod never would. Every per-object depth change this project ever produced was the wearer leaning in, so depth now only counts as evidence when several objects move together, which is a camera move and gets reported once instead of as three false breaks.

Gemini describes the same fact three different ways across takes, "upright", "on napkin", "placed on table", so free text is never compared, only a controlled vocabulary, and "unknown" is treated as no reading rather than as a value. That is what stopped the mug being flagged four times for being honestly unreadable.

Flash-Lite was ten times faster and reported a fully mirrored room as correct, so speed bought by not looking is disqualified on recall. I then made the same mistake myself, turning thinking off after validating against a prompt I had retyped instead of imported. The two texts differed by one sentence, and against the real one the setting caught 0 of 3 mirrored rooms where the harness had measured 3 of 3. A detector that never fires looks exactly like one that works until something is wrong in front of it. There is now a test that fails the build if that setting comes back.

The first version of the rolling view reported HOLDS six times in a row against a closed privacy shutter, because a black frame passes every size check and the model is honest about seeing nothing. Frames are measured for light before they are trusted.

The app also ran for weeks with ClickHouse powering one endpoint out of eight while /api/health announced "json". On a ClickHouse track that is the wrong way round, and it is fixed.

The most expensive one was near the end. The writeup claimed three layers under the question box, and the third was that the cluster credential itself cannot write. I went to demonstrate it, aimed a DROP TABLE at what I believed was a readonly credential, and watched the observations table disappear. The service had been running as the admin user the whole time. Nothing in the code connected the claim to the environment that produced it, so the two had drifted apart silently. The table came back from the JSON state in two minutes, because ClickHouse here is a projection of files on disk rather than the only copy. There is now a real SELECT-only cluster user, it is the one the MCP subprocess connects with, /api/capabilities reports which credential is live so it can be checked from outside, and eight tests fail if the admin credential can reach the SQL agent again.

Accomplishments that we're proud of

On five real three-minute takes of one desk, where the ground truth is that nothing was deliberately moved, a naive extract-and-diff reported 8 differences and 7 of them were fake. Four passes, each added because of a defect measured on this footage rather than anticipated, bring that to 6 findings with no false alarms. The last pass adds findings rather than removing them: giving the model a controlled state vocabulary let it report two things it had been seeing all along and had no way to say.

It costs 1.6 cents per take, measured on real footage through the same Vertex AI path the deployment runs, at Google's published rates. A forty-take day is about 63 cents, or 0.04% of the day it is guarding.

And the question box is a public URL that lets a model compose SQL against a live cluster, which deserved an answer rather than a hope. Asked to ignore its instructions and drop a table, the model declines and attempts no destructive SQL at all. The MCP server's write and drop flags are set off explicitly rather than left to a library default. The credential underneath is granted SELECT and nothing else, and returns ClickHouse code 497 for DROP, INSERT, TRUNCATE, CREATE TABLE and ALTER while SELECT is unaffected.

What we learned

That the honest reading of a field is worth more than a confident one. Treating "unknown" as no reading rather than as a value is the single change that moved this from a tool that cries wolf to one a crew could leave running.

That a safety property nobody can check from outside is a belief, not a property. Both times this project was wrong in a way that mattered, the code and the writeup disagreed and nothing in the system was capable of noticing. Both are now reported by the running service rather than asserted in prose.

And that the hardware decides the algorithm more than the algorithm decides the hardware. Nearly every design choice here, the depth rule, the controlled vocabulary, the light check, the duty cycle, exists because the camera is on somebody's head and moves when they do.

What's next for Dailies

The glasses can speak but cannot listen. The obvious next feature is the operator saying "mark take 3" instead of touching a phone, and it is not buildable on the current toolkit: the public Stream interface exposes video, photo capture, state and errors, and no audio input. The classes plainly exist inside the SDK, so the capability is there and reaching it today would mean binding to internals that any release can move.

Beyond that: a real set rather than one desk, more than one scene at a time, and the continuous capture path, since the SDK soak test already proved the three-minute clip limit belongs to the stock camera app rather than to the platform.

Longer write-up

The full build story, including the measurements that changed the design and the two times the code and the writeup disagreed: https://jonathanandrei.com/blog/dailies-smart-glasses-continuity-clickhouse-agentic-cinema/

Built With

  • android
  • clickhouse
  • clickhouse-cloud
  • cloud-build
  • cloud-run
  • fastapi
  • gemini
  • google-genai
  • javascript
  • jetpack-compose
  • kotlin
  • mcp-clickhouse
  • meta-wearables-device-access-toolkit
  • model-context-protocol
  • playwright
  • pytest
  • python
  • ray-ban-meta
  • uvicorn
  • vertex-ai
Share this project:

Updates