Inspiration
PG&E's Form 10-K, filed with the SEC on 25 February 2021, records that the utility agreed to plead guilty to 84 counts of involuntary manslaughter and one count of unlawfully causing a fire over the 2018 Camp fire, and accounts for a $13.5 billion liability for the victims' claims. The same company's Q2 2020 press release, filed on 30 July 2020, reports: "Situational awareness completion exceeds 30 percent, with 144 weather stations and 60 high definition cameras installed."
So cameras were part of the remedy. Sixty cameras across that territory is thin coverage, though, and a camera only helps if something reads its frames.
What it does
In a single still, cloud, fog, dust and smoke look much the same. What separates them is how they behave over the next ten minutes, so Firstsmoke watches a shape for several frames before it says anything.
- Detect the change. Each camera has a background model with the sun's movement fitted out, and candidate regions are grown by hysteresis so a faint plume top stays attached to its dense base.
- Turn a pixel into a bearing. A detection between the suspect and confirm thresholds converts its pixel column to a compass bearing, and the geometry names the cameras that overlook that ground and where in each frame to look.
- Rank the queue. Agreeing bearings are crossed, with an uncertainty ellipse from
each camera's angular error, and that flag sorts first with a position. A confident
camera with nobody to cross against still reaches a person, marked
NO_SECOND_VIEWand carrying no position. A crossing that cannot be trusted is refused by name, with the nearest approach of the rays. Every flag carries its frames, its bearings, which cameras were consulted and what each answered.
A camera that cannot see — night, fog, rain on the lens, direct sun, a frozen feed — says why, and its silence never counts as an empty hillside. You can also upload a video or stills from one camera.
Held-out results. The threshold (0.45) and calibration were chosen on 64 development sequences from HPWREN's FIgLib and committed before the 130-sequence test set was downloaded. All 130 are scored.
| At the shipped point, 0.45 | Test set, 130 held out | Development set, 64 |
|---|---|---|
| One camera: fires found | 100 of 129 — 77.5% | 50 of 63 — 79.4% |
| One camera: false-positive frames per camera-day | 154.2 | 150.3 |
| One camera: median alert after the human mark | +240 s | +438 s |
| Alerts ahead of the human mark | 0 of 100 | 0 |
| Triangulation: fixes / median spread between pair fixes | 5 of 17 / 6.3 km | 2 of 8 / 7.4 km |
The second camera is not a gate, and we measured why. On the 67 held-out fires that two or more summits recorded, requiring two cameras to agree finds 29.9% of them at 19.6 false-positive frames a camera-day; raising one camera's own bar to 0.55 instead finds 43.3% at 16.2 — more fires for fewer false alarms, from one camera. So corroboration orders the queue and attaches a position rather than deciding whether anything is raised at all. It never alerted ahead of the human mark, on any test fire.
Real footage from outside the network, through the live upload page: a Waldo Canyon time-lapse raised 27 flags, nine before any smoke was in frame, with the highest score (0.75) on the smoke column; a Grand Canyon cloud time-lapse raised 5 flags with no fire present; and a faint prescribed-burn wisp in North Derby Gulch was missed, the top box sitting on a wind-blown bush.
How we built it
OpenCV 5.0.0.93, pinned, because OpenCV 4.14.0 shipped after 5.0.0 and an unpinned
install resolves to 4.x. OpenCV does the work at every stage:
createBackgroundSubtractorMOG2 with shadows discarded, phaseCorrelate with a Hann
window for mast shake, a Sobel threshold sweep and a full-resolution refinement for
the skyline, hysteresis thresholding on the photometric difference, morphologyEx and
connectedComponentsWithStats for candidates, Laplacian variance and HSV statistics
for the way smoke veils what is behind it, and cv2.dnn for an ONNX confirmation head.
Our own tracker, because OpenCV 5 removed TrackerCSRT, TrackerKCF and the
legacy namespace from the main wheel. Appearance trackers lock onto texture and a
plume has almost none, so we associate by overlap of the segmentation masks.
Our own ONNX model, because cv2.dnn in OpenCV 5 loads ONNX only and we found no
licence-clean ONNX smoke classifier: YOLOX is Apache-2.0 but COCO has no smoke class,
and the YOLO models with community smoke weights are AGPL-3.0. So we trained a
53,978-parameter network in plain numpy on pyronear/pyro-sdis (Apache-2.0) and
exported it to ONNX. 0.948 precision, 0.766 recall on that validation split, 0.54 ms a
crop.
AWS. A two-stage container in Amazon ECR on AWS App Runner, eu-west-1, 2 vCPU and 4 GB, with the bundled incidents rendered into the image so a cold start needs no network access.
Evaluation. HPWREN's FIgLib holds recorded sequences from real mountain-top cameras
in southern California, labelled with the seconds from when a person first marked the
plume. We split it into 64 development sequences and a 130-sequence test set, and froze
the configuration in 62063f7 before downloading the test set.
Challenges we ran into
False alarms. The first evaluation found 90.5% of development fires at 485 false alarms per camera-day, concentrated on particular views. Each camera then learnt a nuisance map from its own clear frames on other dates, and the threshold moved from 0.35 to 0.45. On the test set the calibration made no measurable difference — 154.2 false alarms a camera-day with or without it — so the per-view false positives are still unexplained.
The corroboration gate did not earn its complexity. We built the product around requiring a second summit to agree, and the held-out set showed one camera at a higher threshold beating that rule on both detection and false alarms. The loop stayed; its job changed from filtering alarms to ordering them.
Speed traded for quiet. Raising the threshold moved the development median alert from +60 s to +438 s; on test the median is +240 s and the mean +528 s, against published means of 2.3 to 4.7 minutes on the same library.
Triangulation on real data. A fix on 5 of 17 dates where two summits saw the same fire, with pairs of fixes a median 6.3 km apart. The cause is upstream: the bearing comes from the first threshold crossing, and at these false-alarm rates that is often not the fire.
Accomplishments that we're proud of
- A held-out test set, frozen before download and scored in full, reported next to the development numbers where it is worse — including measuring our own central design claim and changing the product when the measurement went against it.
- An agent loop where the pixel column of a detection decides which camera is read next, pinned by tests that place a fire at a known position and check the agent finds it.
- Geometry that is exact where it can be checked: on rendered incidents the crossed bearings land 13 m from the placed fire, inside a reported 161 m ellipse. And refusals with reasons — a named code for an untrustworthy crossing, a named fault for a camera that cannot see.
What we learned
Finding smoke is the easy half; rejecting everything else is the hard half. Our named impostor rules fired 205 times across the 10,107 test frames and the persistent false positives passed every one. A per-camera map that looked like a gain on development did nothing on test, and a corroboration rule we had built the whole product around cost more fires than it saved false alarms. Without the held-out split we would not have known either.
What's next
- Work out what the persistent per-view false positives physically are, since calibration did not remove them.
- Add quiet days to the evaluation. Every FIgLib sequence contains a fire, so the clear period is only the forty minutes before each ignition.
- Fit a per-camera azimuth correction from disagreement with neighbouring cameras, so a re-aimed camera is caught.
Footage credits
Shown in the film:
- Wikimedia Commons,
File:5_Day_Timelapse_-_Waldo_Canyon_Fire_-_June_23rd-28th_2012.webmby Steve Moraco, CC BY 3.0. Excerpt, cropped, re-encoded, original soundtrack removed. - Wikimedia Commons,
File:B-roll_Video_-_Time-lapse_of_Clouds_over_the_Canyon_-_November_20,_2025_(55087231001).webmby Grand Canyon National Park (NPS), CC BY 4.0. - Wikimedia Commons,
File:Prescribed_Burn_in_North_Derby_Gulch_Area,_CGNF_(53229395208).webmby Jamie Balke, USDA Forest Service, public domain (work of the US federal government).
Slide photography is credited on each slide and in slides/IMAGE-CREDITS.txt.
Used without imagery shown:
- HPWREN Fire Ignition images Library (FIgLib), University of California San Diego, http://hpwren.ucsd.edu. CC BY-NC-ND 4.0. Used for evaluation only; no imagery shown.
pyronear/pyro-sdis, Apache-2.0. Training data for the confirmation model; no imagery shown.
No AGPL dependency; nothing imports ultralytics.
Log in or sign up for Devpost to join the conversation.