HgShadow

HgShadow images an absorbing vapor plume as an optical shadow: OpenCV 5 maps optical density and apparent transport, an agent asks a human before any alarm, and AWS Graviton reproduces the run.

Comment

Inspiration

Mercury vapor is invisible and toxic. A point analyzer reports one number at one spot; it cannot show where a plume goes or how fast it moves. HgShadow is meant for people who need to see where an absorbing vapor travels, not only a single reading: researchers and environmental and occupational health teams. We wanted to find out whether a camera can make such a vapor visible as a measurable shadow, and to build that carefully enough that every number says exactly what it is.

What it does

HgShadow is a research prototype for imaging an absorbing vapor plume as an optical "shadow". It compares frames lit at a wavelength the target absorbs (on channel) with a nearby reference (off channel), maps differential optical density and column mass, segments candidate plumes, and estimates apparent transport across a transect with dense optical flow.

A deterministic LangGraph agent reads those OpenCV outputs, confirms candidates with extra frames, recovers once from saturation, asks for a reference measurement when confidence or distance is insufficient, and pauses every alarm proposal for an explicit human approve or reject decision.

Today the evidence is: synthetic data with analytic ground truth, an uncalibrated physical USB camera, an Arduino Uno running LED safety firmware checked on the board, and a complete run on AWS Graviton. Validation is in progress: the two-LED bench is being wired for a dye-surrogate calibration, and mercury validation with a commercial sealed cell and a qualified partner is the next stage.

How we built it

  • OpenCV 5 (5.0.0, checked at every run; other major versions are refused): per-channel optical density and blank-derived thresholds, morphologyEx and findContours segmentation, DISOpticalFlow (Farneback kept as a comparison), VideoWriter, Debevec response recovery, ECC alignment, chessboard lens fitting, undistortion and plane homography, and ArUco markers as an independent motion reference.
  • Agent and human control: LangGraph decision graph (no LLM), FastAPI service with a local web console, SQLite review and evidence stores. Approval decisions are claimed atomically and survive server restarts. Pan/tilt, exposure and range tools are simulated and labeled as such.
  • Hardware: a USB camera with manual exposure verified through DirectShow readback; Arduino Uno firmware that lights one LED at a time, turns off after 3 s of host silence or 20 s on, and boots with outputs off.
  • AWS: one command in CloudShell creates a private S3 bucket, an IAM role limited to that bucket and a security group with no inbound rules, then launches a c7g.xlarge Graviton instance. The instance checks the SHA-256 of every bundled file, builds a pinned Docker image (OpenCV 5 with the Arm NEON baseline), runs the synthetic S0 analysis, two physical camera recordings, eight agent cases, a thread-count benchmark and the full test suite, uploads the evidence and terminates itself.

Results

Synthetic S0 (moving plume with photon and read noise, dust and interferent absorption, independent analytic truth):

Case Column error Flux error, optical flow Flux error, known velocity
DIS, seed 42 4.72 % 2.33 % 0.70 %
DIS, seed 2026 4.73 % 2.59 % 0.72 %
Farneback, seed 42 4.72 % 43.3 % (kept as a documented failure) 0.70 %
  • Agent: eight scripted cases (approve, reject, blank, weak signal, saturation recovery, lamp failure, distance failure, transient). No alarm action runs without a matching approval; six concurrent approvals of one proposal produce exactly one action.
  • Physical camera (uncalibrated): a 10-minute fixed-exposure run kept 21 of 21 manual-mode checks with no clipping and a 0.200 % coefficient of variation. In one motion session the live and replayed results were byte-identical; another session showed a real shadow event with arrows following it, and sporadic false positives in about 2 % of static frames.
  • Arduino firmware: 10 of 10 safety checks passed on the board, with no LEDs attached yet.
  • AWS Graviton versus the laptop: the same code, dependency pins and input hashes; 281 of 281 tests passed on both; every camera frame received the same status (879 of 879 and 585 of 585); the eight agent cases took the same decisions; S0 metrics agreed within 1.5e-7 relative. Graviton wall time was 11 % to 22 % of the laptop's for every step (one run each), and the compute cost is at most about USD 0.0002 per 1000 analyzed frames.
  • Software: 290 automated tests; the scientific functions are checked against independent synthetic ground truth.

Challenges we ran into

  • Farneback flow underestimated smooth-plume transport by 43 %. We kept that failure in the evaluation and selected DIS.
  • The real camera brought problems the simulator did not: exposure settling after start, a black level clamped near code 0, and false shadows at strong image edges. Warm-up, reference stability checks, coverage exclusion and a region of interest addressed them.
  • Reproducibility across machines: a test tolerance was tighter than Linux math-library rounding, and code hashes differed between Windows and Linux because of path sorting and build metadata. We found both by running the same Docker image locally on x86 and emulated Arm before spending anything on AWS.
  • Independent reviews found that no test checked S0 accuracy against the truth; new bounds now catch, for example, a doubled frame interval (about 50 % flux error).

Accomplishments that we're proud of

  • The same decisions on a Windows laptop and on AWS Graviton for every camera frame and agent case, with hashes proving the same code and inputs.
  • Failures and limits are part of the evidence, not hidden.
  • A human approval gate that cannot be bypassed, and hardware that defaults to off.

What we learned

  • Optical flow measures apparent image motion, not gas velocity, and a column map is not a concentration without a measured path length.
  • A consumer camera is not a radiometer: processed MJPG codes need independent response checks before any absorbance is trusted.
  • Reproducibility needs evidence: hashes, labels, pinned images and a second machine.

Responsible design

  • No free mercury, improvised sources or toxic gases. A future UV-C stage requires a closed opaque enclosure and an independent physical interlock; software alone cannot authorize illumination.
  • Invalid, saturated or below-limit data return a status and no number; negative noise is preserved for statistics.
  • Every dataset carries its label: synthetic, surrogate, camera characterization, sealed cell or field.
  • No automatic alarms: every proposal needs a human decision, and no physical alarm is wired.

What's next

  • Wire the two-LED bench, run its first-light check and a tartrazine dye calibration with held-out samples from an independently weighed stock.
  • Measure physical motion truth with a printed marker moved at a known speed.
  • Fix four method defects documented in our limitations, evaluated on new data.
  • Validate with a commercial sealed mercury cell and a qualified partner, inside an interlocked UV enclosure.

Code access and live demo

The code repository is private. Judges have the full code archive (source, tests, documentation and results) and access instructions in the submission's testing information, and can be added to the GitHub repository on request. We are glad to arrange a live screen-share demonstration of the console, the agent approval flow and the camera pipeline.

Built With

Share this project:

Updates

Submission history