Smoke Sentinel

Inspiration

A wildfire is cheapest to fight in its first twenty minutes, and fire-watch cameras are already pointed at the right ridgelines around the clock. The scarce resource is not the camera. It is a human looking at the right one of hundreds of feeds at the right moment.

What it does

Smoke Sentinel scores every frame from a fixed camera for early smoke and escalates to a human dispatcher with the evidence. It never dispatches anything itself.

The key decision is that early smoke is not an object you can draw a box around. It is a faint, low-contrast, slowly spreading change in texture that looks like fog, dust, haze or a cloud shadow. So this is temporal change detection, not object detection: build a reference from the camera's own earlier frames, and measure what is different now.

How we built it

Classical OpenCV 5 throughout: an anchored median reference, findTransformECC drift alignment, a per-channel photometric fit, signed luminance residual, Sobel and boxFilter texture energy, HSV desaturation, morphologyEx masking and connectedComponentsWithStats geometry. Twelve measurements feed a depth-3 decision tree fitted on 35 design cameras. No learned representation appears anywhere, and the model never scores pixels.

Results on 43 cameras never seen during design: AUC 0.846, average precision 0.906, against 0.440 for the naive luminance-residual baseline. Per-sequence median AUC 0.937. At the operating point where all 43 sequences are caught, median detection latency is 420 s from the annotator's "plume visible" mark.

Splits are by camera, never by frame, so no camera contributes to both fitting and reporting. scripts/selftest.py recomputes every published number from committed artefacts in about 20 seconds.

Challenges we ran into

Three bugs, and they turned out to be the same bug three times.

The reference contaminated itself: a sliding baseline window crept forward with the query frame until it was 83% post-ignition, so the detector was subtracting the plume from itself, worst exactly when the fire was largest. Anchoring it took AUC from 0.50 to 0.85.

Per-scene offsets swamped the signal: features scored 0.67 to 0.73 AUC within a sequence but 0.53 pooled, because an absolute band-coverage number means different things on a hazy valley camera and a clear ridge. Per-camera median/MAD calibration fixed it.

Then the agent's own history pool turned out to be contaminated the same way, which is the next section.

The agentic layer, and the honest result

OpenCV 5 output drives a multi-step perception, decision and action loop: the detector's measurements decide whether the agent requests a second look, which neighbouring camera and which historical comparison it pulls, and whether a candidate is escalated to a human or suppressed. The loop is diagrammed in docs/agent-workflow.png and every decision is traced in the committed artefacts and shown live in the demo interface.

We built it specifically to cut the false-alarm rate. Measured on held-out cameras, it removed 7.0% of false alarms at a cost of 9.3% of recall, missed the bar we set beforehand, and is therefore disabled in the reported system. We are reporting that instead of quietly dropping it. The diagnosis generalises: every verification step was reading the same measurement the detector had already read, so a frame the detector got wrong was one the loop got wrong too. A second look needs a second kind of evidence.

The demo video shows a real suppression that was wrong, with its trace.

Accomplishments we are proud of

Honestly, that negative result, written up in full in docs/report.md section 7 rather than buried.

We are also proud that the headline is a limitation. At the operating point where every fire is caught, this system raises about 86 false alarms per camera-day, and a real fire-watch desk needs well under one. That gap is stated in the first paragraph of the README, not hidden at the bottom.

What we learned

Any time a system compares "now" against "recently", on data where the event of interest persists, the reference is a candidate for contamination. We found that bug class three times in one project, and the third time only because we instrumented the thing we assumed was fine.

AWS: designed, never deployed

infra/ holds the SAM stack, Dockerfile and Lambda handler, and docs/deploy.md explains the design. It was never applied. The $150 compute-grant round closed before we could enter it, and we were not willing to attach a personal payment card to a cloud account for a student competition. There is no stack, no endpoint URL, no Graviton numbers and no cost figures anywhere in this entry, because we never ran any. The rules offer a working endpoint or an arranged live screen-share, and we are taking the screen-share.

COOL was not used at all, so the COOL award is not claimed.

What's next

Rebuild the camera-neighbour map over the design split so the verification policy can be tuned without touching the reported numbers, then give the loop a genuinely independent second signal: a vision-language model reading the candidate crop, a wind-direction prior, or a satellite hotspot feed. Per-camera threshold calibration from each camera's own quiet hours after that.

Data

HPWREN FIgLib fire-watch imagery, used for research. The pipeline stores per-frame measurements, not imagery. Output is a review queue for a human dispatcher: no automated dispatch, no decision about any individual.

Built With

• aws-lambda
• aws-sam
• numpy
• opencv
• playwright
• python
• scikit-learn
• strands-agents
• uv