AeroNav

Inspiration In a burning or collapsed building, rescue teams can't easily see inside, and every minute spent searching blind costs lives. Small, everyday losses have the same problem: a lost phone or set of keys in a big area is hard to find by walking back and forth. We wanted to see whether a small, cheap drone could search a space by itself (with no map and no GPS) and use AI to recognize what it's looking for.

What it does AeroNav is a website with two autonomous drone missions, both running in the browser:

Object Finder: You mark an area of a city street by clicking four corners and pick what was lost: wallet, keys, phone or purse. The drone plans a lawnmower-style route, photographs the ground, and goes back to spots that passing cars or pedestrians blocked. Every photo goes through our AI model, and the item is marked on the map with a red beam you can click to see the photo.

Rescue Mission: A four-storey building is on fire with three people trapped inside. Two drones split the building between them and search it on their own, finding their way with simulated sonar. An AI person detector checks their camera images. A person counts as found only when the AI sees them from two different positions with high confidence. You have 20 minutes of simulated time before the building collapses, and you can run the mission at 1×, 5× or 10× speed.

How we built it Simulation: Unity 6, built for WebGL so it runs in a browser. The buildings, furniture, people and fire are generated from a random seed, so every mission is different. The same building can be replayed for testing. Navigation: The drones aren't given a map. Each one sends simulated sonar pulses, builds its own map of obstacles from the echoes, and replans its route when it detects something new. AI: Object Finder uses a YOLO11n model we fine-tuned on Unity renders of the city, for wallets, keys, phones and purses. Rescue Mission uses a YOLOv8m-worldv2 person detector, also trained on renders of our building, exported to ONNX and run locally with onnxruntime. Both run on our own machine; no cloud API is involved. Backend: A Python server serves the website, stores every drone photo, sends it to the model, and returns the detections. Locating people: For each detection we cast a ray from the drone camera's exact position through the center of the box into the 3D scene, which gives an estimated floor and position. Searching: A spot only counts as searched when a camera frame actually covered it. The drones turn the camera toward the most unsearched area, stop at good viewpoints to look, and keep searching until everyone is found or time runs out. Challenges we ran into "Searched" didn't mean "seen." At first a spot counted as searched whenever a drone flew near it, even if the camera was pointing the other way. Some people never appeared in a single photo. We rebuilt coverage around what the camera actually saw. Distance matters a lot. Our tests showed the person detector finds people about 93% of the time at 1–2 m, but only about 35% at 4–5 m. We changed the drones to inspect spots from closer up. False alarms. The model sometimes mistook a toilet for a person. We required two separate sightings and raised the confidence threshold based on measured data. Fire trapped the drones. Spreading fire could block the only route to the stairs, leaving a drone hovering in place. We gave the drones an escape behavior and tuned the fire speed so missions are winnable. Speed vs. AI. At 10× the drones moved faster than the AI could process images. We made the simulation slow itself down automatically when the AI falls behind. Browser limits. The city crashed Chrome with an out-of-memory error until we cut the number of simulated pedestrians. Accomplishments that we're proud of Real AI inference runs end to end. No detection is faked, and the people's true positions are used only to score the result, never shown to the AI. In our controlled tests the Rescue Mission regularly found everyone within the time limit. In earlier browser tests of Object Finder, the drone found the hidden item in nearly every run, with a position error of only a few centimeters. The drones navigate purely by sonar, with no map. We built automated tests for the search, the AI model and the mission outcomes, and fixed several problems because those tests caught them. What we learned Measure before tuning. Increasing the frame rate didn't help until we measured where survivors were being missed. An AI model is only as good as the images you give it. Camera angle, distance and lighting mattered as much as the model itself. In a search, "covered" has to mean "seen by the sensor," not "passed nearby." Building a simulation with repeatable seeds made it possible to compare changes fairly. What's next for AeroNav Search the rooms nearest the fire first, before they're cut off. Train the person detector on harder cases: darker rooms, people lying down, people partly hidden, and objects like toilets that confuse it. Let more than two drones cooperate and share their maps live. Move from simulation to a real small drone with a camera and distance sensors. Send rescue teams a live map of where people were found.