Architecture

Inspiration

Most of the world's rivers have no gauge. The small, fast rivers that cause flash floods are usually unmeasured, because a gauging station costs tens of thousands of dollars and needs regular in-river measurements.

A camera can measure a river. Image velocimetry tracks the water surface between frames and turns pixel motion into metres per second, and, with a surveyed cross-section, into discharge. What stops it running unattended is judgement. The same code that is accurate on a good clip returns a confident wrong number on a shaky, flickering or texture-less one, and never says so. In our baseline, three of four standard methods returned near-zero speeds (−95 to −100% error) on unseeded water with no warning.

The authors of the main river-video dataset list exactly this as future work: metrics to judge whole videos and eliminate poorly seeded ones. Gaugeless is that missing judgement.

What it does

Gaugeless takes a river clip and runs: frame QA → diagnose → repair → measure → check → publish, refuse, or escalate to a human.

Diagnoses the clip: tracer density, camera shake (a flow-jitter score), repeated frames, flicker, glare.
Repairs what it can, and checks each repair: stabilisation is kept only if it actually reduces camera motion. It also de-duplicates frames, masks glare and normalises brightness.
Measures surface velocity with four OpenCV methods (KLT-IV tracking, LSPIV, STIV, DIS) and turns it into discharge at surveyed cross-sections.
Checks the physics: two-section continuity (the same water passes both sections, so their flows must agree), the Froude number and coverage. Continuity needs no reference data, which makes it the trust signal for a river nobody has gauged.
Refuses untrackable footage instead of guessing.
Never sends a flood alert without a human. Escalations go to an approve/reject email, and the tools make it impossible for any planner, including an LLM, to skip that step.

Results on real rivers (Perks et al. 2020 dataset, 7 countries, in-river reference measurements):

Site	Camera	Error vs reference discharge
La Morge (FR)	pole camera	−0.4%
Thalhofen (DE)	fixed camera	+1.0%
Murg (CH)	4K drone	+3.4%
La Vence (FR)	smartphone	−10.2%
Castor (CA)	raw oblique video (we orthorectify it ourselves)	−13.8%

On 105 damaged real clips (shake, flicker, blur, glare, repeated frames), a fixed pipeline published 24 silently wrong numbers; the agent published 0. Under 10 px of shake the fixed pipeline's error reached 179–472%; the agent stayed at 12–24%.

How we built it

OpenCV 5 does all the vision work:
- velocimetry: calcOpticalFlowPyrLK + goodFeaturesToTrack (KLT-IV filters), matchTemplate (LSPIV), FFT space-time images (STIV), DISOpticalFlow;
- camera solve from ground control points: calibrateCamera, projectPoints;
- water-plane orthorectification: remap;
- grid registration: SIFT + estimateAffinePartial2D;
- stabilisation: RANSAC similarity transform on bank tracks;
- a local LLM running inside cv2.dnn (ENGINE_NEW, KV cache, cv2.dnn.Tokenizer).
The agent has 13 tools with guard rails and three interchangeable planners: deterministic rules, Claude Sonnet 4.5 on Amazon Bedrock (Converse tool use), and Qwen2.5 inside OpenCV's own DNN engine. The same tools are exposed through an MCP server, which has no approve tool, so an agent can't approve itself.
AWS, deployed and tested live: S3 upload → SQS → Graviton (arm64) EC2 worker → DynamoDB. Escalations go to Step Functions (waitForTaskToken) → Lambda → SNS approve/reject email, and only an approved job sends the alert. CloudWatch dashboard and budget alarm. The whole stack is one CloudFormation template.
A web app, live on AWS at http://100.63.108.38/ (on the Graviton worker): pick a river, damage the clip, and watch each agent decision stream in by phase (perceive, repair, measure, check, decide) next to the velocity overlay. With the Claude planner its reasoning streams too. Escalations stop at Approve / Reject for the operator.
COOL on Graviton3: we benchmarked the same code on stock OpenCV 5.0 and on the Cloud-Optimized OpenCV Library AMI (c7g.large, interleaved runs, 5 repeats, 4 rivers). The agent ran 1.16× faster end to end (KLT tracking 1.18×, stabilisation 1.10×; STIV was slower at 0.75×) and made the same decisions on all 5 test clips, with La Morge's discharge identical. Caveat: COOL ships OpenCV 4.12, so the speed-up mixes its build with the version difference.

Challenges we ran into

Silent failures everywhere. No single method wins; each fails silently somewhere. We built the evaluation around counting silent failures, not average accuracy.
Our own repairs made things worse at first. The evaluation caught each case:
- LSPIV gap-filling made three rivers worse;
- a stabiliser accepted a bad fit and doubled Murg's error;
- a detrend step silently changed a clean clip's speed. Each repair now checks itself.
Shake fooled the physics check. On La Morge, 10 px of shake gave an 87% error that still passed continuity, because shake corrupts both sections alike. We added a flow-jitter detector and flow-based stabilisation for clips with no visible bank.
Raw oblique video (Castor). It needed a full camera solve with lens distortion. A plain pinhole model was 16 px off, and ours is 2.5 px. The published reference grid also turned out to be mirrored.

Accomplishments that we're proud of

Discharge within 0.4–14% of in-river measurements at 5 real sites, with the textbook velocity index (not fitted to any site).
Zero silent failures across 105 damaged clips, against 24 for the fixed pipeline.
Claude on Bedrock made the same publish/refuse decision as our hand-written rules on 14 of 14 rivers, with no site-specific code. When it once tried to publish without measuring the gauge points, the tools refused, and that check is now a permanent guard rail.
A live AWS deployment where a real flood-threshold job waited for a human, and both a rejection and an approval were recorded end to end. The dashboard is public at http://100.63.108.38/.

What we learned

Small local LLMs (Qwen2.5 0.5B and 1.5B) run well inside OpenCV 5 (14–22 tokens/s on a laptop CPU), but they can't plan this task: 1–3 of 6 correct decisions, and they miss "refuse an unseeded clip". We use them for offline field reports and let Claude plan.
Guard rails in the tools, not in the prompt, are what make an LLM planner safe.
An honest "I can't measure this" is worth more to a flood operator than a confident guess.

What's next for Gaugeless

Field validation on a stream we film ourselves (phone, tape measure, float timing).
Night and rain footage.
Per-site velocity-index calibration.
Estimating cross-sections from bank geometry for sites with no survey.
Running the worker on COOL once it ships OpenCV 5.