Sightline: Titlecard
Survey: 500 aerial photos from a simulated drone search over Mount Seymour.
Rank: AI scans every photo and brings likely sightings to the front.
Review: A searcher verifies candidates while the probability map updates live.
Search next: The heatmap highlights priority areas for ground teams.

Sightline

Finds the hiker in hundreds of search-and-rescue drone photos, then shows the rescue team where to look next.

Inspiration

North Shore Rescue already flies drones over the mountains. The slow part is a person scanning hundreds of 12-megapixel photos for a hiker who is a tiny smudge in each. The published human baseline (Pyrrö, Naseri & Jung 2021) is 33 seconds per photo, with 68% recall and 59% precision: slow, and about a third of people missed. We wanted to make that review faster, and to use the photos that show nobody as information too.

What it does

The site walks through one survey in five steps:

Survey: 500 drone photos from a search over Mount Seymour.
Rank: a detector puts the likeliest people first in a review queue.
Review: a searcher answers each candidate (Person, Not a person, Unsure); answers become map pins.
Search next: a map of where the person could still be, updated after every photo.
Results: the numbers, with sources.

A "Person" answer is a sighting to check, not a confirmed find (it could be a searcher or another hiker). Sightline never decides on its own that an area is clear: the map suggests, rescuers decide.

A second survey uses real HERIDAL photos at their recorded GPS positions. The Mount Seymour flight path is simulated, and the site labels it that way.

How we built it

We split the work into four lanes that meet at one shared data contract. Heavy work runs once, offline, on a GPU desktop and writes a folder of JSON and images. The website only reads that folder, so the demo needs no GPU or backend.

Lane A: Detector. We fine-tuned YOLO11s for 80 epochs on 1024 px tiles cut from 890 labelled HERIDAL training photos. At run time, SAHI slices each full photo into tiles and merges the boxes. The 0.29 threshold was chosen on the validation set to favour recall, because a missed person costs more than a false alarm a searcher dismisses in seconds.

Lane B: Geolocation and map. We read camera position and angle from photo metadata, work out the ground each photo covers, and turn each detection into a map pin. The Mount Seymour demo places real HERIDAL photos on a simulated flight path. Pins use a 10 m position radius, which is also simulated. The map uses MapLibre with an offline map file as a fallback.

Lane C: Bayesian search map. The key idea: photographed is not the same as searched. An empty photo of open ground is strong evidence the person isn't there. An empty photo of forest is weak evidence, because trees can hide them. So each photo lowers the probability more where the camera could see well, and the probability shifts to places it couldn't see.

A three-cell example. Before the photo: meadow 0.50, forest 0.30, unseen 0.20. The chance the photo would have spotted someone there is 0.80 for meadow, 0.20 for forest, 0 for unseen. After a photo that shows nobody, the map becomes meadow 0.19, forest 0.44, unseen 0.37. The forest is now the most likely cell, even though the drone flew over it.

A simple colour-and-texture classifier estimates tree cover in each 5 m cell. The map is built offline, then replayed live in the browser by a TypeScript copy of the same maths, so a "Not a person" answer updates the map on the spot. The starting map (the "prior") is a uniform placeholder, labelled as such.

Lane D: Pipeline, review queue and web app. An eight-stage Python pipeline runs every lane's code and is the only thing that writes the data folder, validating it against the contract first. The React app has a keyboard-driven review queue (P, N, U, undo) wired to the map.

Challenges we ran into

Tiny people. A person is about 70 px wide in a 4000×3000 photo. Shrinking the photo to the model's input size erases them: run on a photo shrunk to 1024 px, recall fell to 0.234. A model trained on whole photos shrunk to 1920 px reached 0.804 recall, against 0.864 for our tile-trained model. Working at full resolution is what mattered.
A label bug. While studying the misses, we found four test photos whose label files listed the wrong image size, squashing 42 person boxes. Fixing it moved recall from 0.769 to 0.864. The model was fine; the score was wrong.
Cell size. Our first grid used 20 m cells, but one photo covers only 20 × 15 m, so a cell mixed open ground and forest. Moving to 5 m cells (40,000 cells) let the tree-cover estimate show through.
The tree-cover classifier is rough. Its average error is 0.34 against 20 hand-scored crops. It misses pale, autumn and dry trees, reads dark lawn as shadow and bright textured rock as trees.
Radius bugs. Checking only cell centres against a 10 m radius would miss every cell for about a fifth of positions, so we measure distance to the whole cell. Rounding errors also dropped neighbours exactly 10 m away until we added a tiny tolerance.

Accomplishments that we're proud of

On the HERIDAL test set (101 photos, 337 people), using the human study's loose scoring: recall 0.896 and precision 0.898, against 0.68 and 0.59 for humans, at 0.25 s per photo (GPU) against 33 s. Under strict scoring: recall 0.864, precision 0.802, AP50 0.868.
Ranking: all 6 demo photos with a person landed in the first 7 photos of the review queue.
Search map (simulated, 5 m cells): a ground team checks a median of 1,213 cells with our map against 1,743 with a naive "photographed = cleared" map (seed 7, the best of three seeds). Across 15 robustness runs it won every time: 28–30% fewer cells at default noise, 19–23% with noise matching the classifier's measured error.

What we learned

Resolution beats cleverness for tiny objects.
Check your labels before trusting your score.
Keep the test set for the end. Our disclosure: the threshold and inference settings were chosen on validation, but two earlier model choices (keeping training run 2 over run 1, and dropping a hard-negative fine-tune) were made with test scores in view, so the test numbers are slightly optimistic.

What's next

A real starting map from lost-person behaviour research instead of the uniform placeholder.
A better tree-cover classifier, especially for pale and autumn trees.
Finishing the site-held-out retrain, since recall is lower on terrain the model never trained on.
Measuring real position error on real flights, and asking "does this match the subject description?" for each sighting.