SARchlight

GIF
Terrain-aware SAR heatmap highlighting priority search zones and possible detections
Live drone video feed detecting a possible person in a dense forest search area.
Workflow

Inspiration

In wilderness search and rescue, time is the enemy. Ground teams cover terrain slowly, aerial spotters get fatigued, and a person under canopy can be almost invisible from above. Even when drones are available, teams still need to answer the hard operational question: where should we look next?

We wanted to build something more useful than another object detector. A detector can say "maybe person in this frame." Search teams need a system that combines that evidence with terrain, search coverage, missed detections, and uncertainty. SARchlight is our attempt to build that missing decision layer: a drone-side search brain that keeps updating the probability of where the person is and uses that map to drive the next move.

What it does

SARchlight gives SAR teams a live search brain:

Builds a probability map from last-known position, terrain, land cover, and accessibility.
Sends drones toward the highest-priority search sectors.
Converts image detections into ground locations with a georeferencer.
Updates the map from both detections and non-detections.
Coordinates multiple drones so they do not search the same area.
Declares a subject located only after persistent, concentrated evidence.
Shows the mission state in a live React dashboard.
After locating the subject, computes a terrain-aware route and simulates a drone guiding them back to operators.

The core loop is:

prior map -> search path -> drone observation -> Bayesian update -> next search target -> located event

How we built it

Search brain

The backend uses a Python and NumPy probability grid. The brain owns a single-writer MapState and applies Bayesian updates as observations arrive.

Terrain-aware prior

The prior is shaped by the last-known position, elevation, land cover, and accessibility. The real-terrain showcase uses Marin County terrain with DEM hillshade and land-cover-derived visibility.

Georeferencing

The georeferencer translates detector boxes from image space into ground cells. This lets the search map understand where a detection happened and which ground cells were actually searched.

Detection adapter

The detector is swappable. The deterministic demo can run on simulated detections, while the integration path supports YOLO-style detector outputs through an adapter.

Multi-drone planner

The planner overlays coarse sectors on the fine belief grid, ranks sectors by probability and remaining coverage, and assigns drones to disjoint sectors.

Dashboard

A FastAPI server projects the brain state into a React, TypeScript, Vite, and Tailwind dashboard with probability heat, drone positions, detections, search path, confidence, locate status, and guide-home routing.

Voice Layer

The voice layer uses Deepgram for both the subject broadcast (text-to-speech) and a Twilio-backed operator phone agent (Deepgram Voice Agent) deployed on Fly.io, which reads the live search state over a dedicated endpoint and answers operator questions through function calls.

Reliability

For reliability, we instrumented the entire stack with Sentry — the FastAPI brain, the React dashboard, and the remotely deployed phone agent — so failures that would otherwise be invisible (a background search-thread crash, a degraded voice line, or an error mid-phone-call) surface immediately; it's fully env-gated, so it adds observability without adding any risk to the core loop.

Challenges we ran into

Making non-detections useful

Search is not only about what the drone sees. Empty ground matters too. We had to model clean passes as evidence without letting repeated misses unfairly erase a subject hidden by canopy.

Correlated misses under canopy

In real terrain, repeated looks from the same sensor can fail in the same way. We added capped cumulative clearance per sensor so one drone repeatedly missing the same obscured cell does not drive that cell to zero probability.

Avoiding premature locate alerts

A single false positive should not trigger a rescue declaration. RescueLoop waits for persistent evidence and probability concentration before declaring the subject located.

Real terrain was harder than synthetic terrain

The real Marin terrain prior made the find more realistic and less forgiving. We redesigned the scenario around a plausible subject cell, canopy visibility, thermal corroboration, and a generalized flight path.

Keeping the system integrated

The detector, georeferencer, brain, server, dashboard, and voice layer all use different data shapes. We kept clear contracts and adapters so the pieces could connect without rewriting each other.

Accomplishments that we're proud of

Built a working closed-loop SAR search system.
Ran the brain on real terrain data.
Located the planted subject at 0-cell error in the real-terrain showcase.
Built multi-drone sector assignment with no overlapping search sectors.
Built a live React dashboard for operator situational awareness.
Added guide-home routing after the subject is found.
Kept the demo honest about what is simulated, what is real, and what still needs field validation. -Deployed a live operator phone agent that answers questions from real-time search state. ## What we learned The most important signal in search is not always a detection. Sometimes it is a well-modeled miss.

We learned that the strongest AI system here is not a single model. It is the loop connecting perception, geography, probability, planning, and communication. A weaker detector inside a strong search loop can be more useful than a stronger detector with no operational reasoning around it.

What's next for

Next, we want to turn Untitled from a software demo into a field-ready rescue assistant.

We would connect it to live drone telemetry, real GPS/camera data, and thermal footage so detections can update the map in real time. We also want to improve the probability model with real SAR behavior data, terrain difficulty, trails, and last-known-position patterns.

Long term, Untitled could coordinate multiple drones, support multilingual voice broadcasts to missing people, and use phone-signal or beacon data as another clue.

Our goal is simple: help rescue teams search faster, avoid duplicated effort, and make better decisions when every minute matters.