TrailWatch: an agent that triages camera-trap footage

An OpenCV 5 agent that investigates every camera-trap trigger, calls a second model when its tools disagree, and holds every alert for a person. 93.5% correct on real data vs 71.9%.

Comment

Inspiration

Camera traps record thousands of triggers a week. Most are wind or empty frames; a few really matter, like an elephant walking towards a village at night. I build Wildlife Network (wlnw.org), an open live map of wildlife detections, and our first field stations go up in Chennai, India, in December 2026. I saw that fixed "AI says X" pipelines fail quietly on exactly the cases that matter: animals that stand still, dark infrared frames and species that look alike. I wanted a system that investigates each trigger the way a careful person would, and never sends an alert without a person agreeing.

What it does

TrailWatch is an OpenCV 5 agent that triages camera-trap footage (a video clip or a burst of stills) and decides what should happen next:

  • log a confident identification,
  • send it to human review when unsure or when the top answer is a look-alike of a watch-listed species,
  • discard empty frames, people and vehicles (not kept, for privacy),
  • or raise an alert that waits for a person to approve it for watch-listed species such as elephants and leopards.

OpenCV 5 measurements decide the next tool call. No motion? Lower the threshold, then ask MegaDetector, which finds animals that stood still. Dark or blurry frame? Enhance it (adaptive gamma, CLAHE, unsharp mask), re-measure, keep whichever version the classifier is surer about. Classifier unsure? Try the enhanced crop, then the next sharpest frame. Classifier says "blank" or is unsure about a loose motion box? Ask the detector for the animal's own box and re-crop. Every step is returned as a trace of tool, inputs, observation and decision.

How we built it

  • OpenCV 5 (5.0.0.93) for all perception: MOG2 background subtraction for video, a median-background colour difference for bursts of stills, morphology and contours, Laplacian sharpness, HSV infrared test, LUT gamma, CLAHE and unsharp masking.
  • Google SpeciesNet (v4.0.3a) to identify the crop, and MegaDetector v5a as the slower second opinion.
  • A Python agent loop with an explicit policy (accept and review thresholds, a watch list, a look-alike rule) and a full trace.
  • FastAPI endpoint with a demo page and six real sample bursts.
  • AWS: Docker on an EC2 Graviton instance (t4g, arm64), with both models running on the Arm CPU at about 2 seconds per burst.

Evaluation

260 real labelled bursts from Snapshot Serengeti (LILA BC), 12 species plus empty, 20 each. Same models in both columns; the baseline is a fixed single pass.

Fixed single pass TrailWatch agent
Correct action 71.9% 93.5%
Animals wrongly discarded 68 of 240 8 of 240
Elephants, lions, leopards alerted 37 of 60 52 of 60
Watch-listed animals reaching a person 40 of 60 56 of 60
False alerts 0 0

Failure cases are in the report: 7 of 20 leopards were read as cheetahs (4 still reached a person through the look-alike rule), and 8 small or hidden animals were still missed.

Challenges we ran into

  • Camera-trap bursts have only three stills, too few for MOG2 to learn a background, so I added a median-background difference that ignores exposure changes.
  • Animals that stand still produce no motion at all; the fix was letting the agent decide when the expensive detector is worth calling.
  • Enhancement sometimes made the classifier worse, so the agent compares raw and enhanced crops instead of trusting either.
  • Human images are withheld from the public dataset for privacy, so people are handled by MegaDetector's person class and are not part of this evaluation.

Accomplishments that we're proud of

Cutting animals wrongly thrown away from 68 to 8 on real data, with no false alerts, and a trace that shows exactly why every decision was made.

What we learned

The biggest gains came from letting cheap OpenCV measurements decide when to call a slower model, not from a bigger model.

What's next

TrailWatch is planned for Wildlife Network's first field stations in Chennai from December 2026 (the Madras Crocodile Bank Trust has agreed in principle to host pilot stations), with Indian watch-list species (elephant, tiger, leopard, mugger crocodile), thresholds tuned from a reviewed sample, and expert validation of the data.

Links

Built With

Share this project:

Updates

Submission history