Thalweg: a camera is a stream gauge

OpenCV 5 turns river video into discharge (m³/s). A policy-gated agent publishes only what the evidence supports, refuses the rest, and drafts flood warnings only a person can sign.

Comment

Inspiration

Floods hurt more people than any other weather hazard, and the measurement that tells a forecaster how much water is coming, discharge in m³/s, still depends on people standing in rivers with current meters and ADCP boats. Video gauging (LSPIV, STIV) removes the danger: a camera, a few surveyed points and a cross-section are enough. What it does not remove is the operator. In the Video Globe Challenge 2020, 15–23 hydrologists measured the same eight videos; on a typical video their median error was 10–23%, and parameter choices were the main reason (Bodart et al. 2023). We built the operator.

What it does

Thalweg reads a river clip with OpenCV 5, measures the surface velocity three independent ways, and computes discharge over the surveyed cross-section. An agent then decides whether the number can be trusted: it publishes, holds and asks for a better clip, or escalates to a hydrologist. Published gaugings refit the station's rating curve and feed a stage nowcast. When a flood threshold is likely within three hours, the station drafts a warning that only a person can sign.

How we built it

  • Camera check and stabilization: ALIKED + LightGlue (OpenCV 5 features), robust similarity on the banks, forward-backward Lucas-Kanade on every frame.
  • Orthorectification: GRP homography or DLT + solvePnP (fitted intrinsics and distortion where needed), remapped onto the water surface at the live water level.
  • Diagnostics: moving-tracer energy, static texture (riverbed visible through clear water), glare, darkness, blur, frozen frames.
  • Velocimetry: LSPIV with ensemble cross-correlation, STIV with sub-pixel shift-and-add, DIS optical flow; velocity-area discharge with an uncertainty budget.
  • Agent: a deterministic autopilot or an LLM with tool calling and vision. Every call passes a policy gate written in code; publishing requires a basis the code verifies against the measurements; every failure holds or escalates.
  • AWS: S3 ingest, EventBridge, Step Functions (activity worker plus waitForTaskToken for the human signature), a Graviton4 c8g.large running the COOL AMI, DynamoDB, API Gateway + Lambda, CloudWatch dashboard and alarms, an AWS Budget, all in CDK.

Results (all reproducible from the repo)

  • Blind benchmark (git tag eval-freeze, videos 2–7): the frozen autopilot published 5 of 6, with mean absolute error 13.1%, better than the median expert on 3 of 6 and ahead of 52% of experts on average. The LLM agent was best of all 19 answers on one video (−1.6%) and landed inside the reference uncertainty on 3 of 5.
  • Refusal suite: zero wrong publications in every run. v1.1 gets 9 of 9 decisions right.
  • Performance: one gauging takes about 38 s on Graviton4. That is about 15% cheaper than x86. On this workload COOL is not faster than the stock OpenCV 5 wheel, which already ships KleidiCV, and we report it as measured.
  • Live station: the River Dart replaying Storm Callum (12 Oct 2018) through the real cloud pipeline, every 10 minutes. On the rising limb 6 of 7 gaugings fall within ±9% of the published camera analysis of the same clips (KLT-IV, Perks 2024); high flows read 24–35% lower. The station drafted the flood warning from its own rating curve and nowcast, and a draft was signed live in the review queue.

Challenges

  • Clear shallow water shows the riverbed, which pins correlation methods to zero.
  • Hand-held footage drifts hundreds of pixels.
  • The River Dart camera needed its focal length and lens distortion fitted from nine collinear-ish control points.
  • A 128 KB disk readahead made a 39 GB archive extract at 6 MB/s.

What we learned

The hard part of video gauging is not the correlation; it is knowing when not to trust it. Making refusal the product, with a verifiable basis for every published number, changed how we built everything else.

What's next

  • Our own vectorized LSPIV/STIV kernels: they are the two slowest stages.
  • Stage read from the image.
  • A per-station reference-frame declaration for hand-held surveys.
  • A pilot on a real IP-camera network.

For the judges

Built With

Share this project:

Updates

Submission history