Inspiration
Search-and-rescue operations are often limited by time, manpower, and the cost of specialized equipment. Commercial thermal drones cost thousands of dollars, so many volunteer teams and small departments have one at most, or none. We wanted to find out how far cheap hardware plus AI could close that gap. Our target was a thermal search drone for under $500 that could do real search work, so a team could afford several instead of hoping one is enough.
What it does
VIGIL is a low cost thermal search-and-rescue drone prototype designed to help operators locate people when visibility is limited. The drone cost under $500 to build, which is nearly 1/10th the price of the nearest competitor.
It combines a thermal camera, person-detection AI, and a ground-station dashboard. Operators can watch live thermal footage with tracked person detection, review each suspected person using the exact image that triggered the detection, and see estimated locations when enough telemetry is available.
Operators can also add or remove person markers manually and create search routes. Locally saved maps and terrain elevation data allow prepared mission areas to remain available without internet access.
How we built it
Lepton 3.5 → ESP32-S3 (Wi-Fi) → ground computer (YOLO26 detection + tracking) → Socket.IO → web dashboard
Flight controller (INAV) → ExpressLRS → radio → telemetry bridge → web dashboard
We built VIGIL as four connected systems: flight control, thermal capture, AI processing, and the operator interface.
The drone uses a custom carrier PCB flight board built around a STM32 Black Pill running a fork of INAV which we modified to work with our chosen sensors. We added support for the ISM330DHCX IMU, a BMP581 barometer, and the LIS3MDL magnetometer. Additionally, we added NMEA GPS support as well as a flight data recorder. ExpressLRS was used to communicate with the drone remotely from the controller, while a Python bridge decodes flight telemetry from the serial output of the controller.
A FLIR Lepton 3.5 thermal camera connects to a Seeed Studio XIAO ESP32-S3 running C++ firmware. The ESP32-S3 captures the 160 x 120 frames from the camera and streams them over Wi-Fi through its own access point.
The ground computer connects to the hosted access point then pulls the stream in Python. On that computer two YOLO26 detectors with an extra high-resolution detection layer for small targets run on every frame:
- a far-range model, trained on drone thermal footage
- a close-range model, fine-tuned on close-up thermal images degraded to look like real Lepton output (motion smear, sensor noise, auto-contrast)
Their detections are merged per frame and tracked with ByteTrack, so people keep a consistent ID. A track is only shown once it has been seen in several frames. Because every pixel carries a real temperature, we also reject "people" too hot to be human, such as lamps and electronics. Our training pipeline uses whole-video validation splits and data-leakage checks.
The dashboard uses Next.js, React, TypeScript, Tailwind CSS, Leaflet, and Socket.IO to bring together thermal review, mapping, telemetry, manual markers, and route planning.
Challenges we ran into
Low-resolution thermal detection was a major challenge. Distant people can occupy only a few pixels, and the Lepton's slow sensor smears anyone who moves. Models trained on clean datasets lost confidence as soon as someone walked, so we fine-tuned on synthetically blurred, noisy frames. Hot objects were the next problem: a shorted circuit board was once detected as a person, which led to the temperature check. Overlapping boxes followed, with one person detected as separate head and body boxes; we fixed that with custom merging rules.
Reliable camera capture required careful handling of packet synchronization, memory buffers, and network delays. We patched the camera library and separated frame acquisition from streaming.
Getting a 7 inch drone to hover stably. Our first takeoff flipped instantly. This added a 12 hour delay to our progress right in the middle of the event. We traced it to a gyroscope high-pass filter in INAV's driver for our IMU that removed the bias a second time after calibration, leaving a constant false roll. Later flights were better, but still oscillated and crashed. We set up proper logging, which due to RAM constraints could only record the last 16 seconds of flight. That logging allowed us to find that our motors had a slight later lag time (40-60 ms) to inputs than most drones which caused the drone to always respond to the current state too late. After that breakthrough we adjusted our gains and were able to achieve a stable hover.
Mapping detections required combining each image with the drone's position, attitude, heading, camera geometry, and terrain. Small convention mismatches matter. INAV reports pitch with nose-down as positive and altitude relative to home, which was not what the CRSF convention implies. We found both by reading the firmware source, and either would have misplaced every pin. Missing or stale information must prevent an estimate rather than produce a misleading pin.
Offline operation required saving both map imagery and numerical elevation data.
Accomplishments that we're proud of
We're proud of connecting custom hardware, thermal AI, and a practical operator interface.
Our documented camera bench testing achieved approximately 8.8 unique thermal frames per second at native resolution. We developed a custom INAV target and achieved a stable hover, with roll and pitch held within about 1 degree. The drone streams attitude, barometric altitude, GPS position, and a calibrated compass heading to our dashboard over ExpressLRS.
On live Lepton footage, our blur-trained model detected the person in 94% of frames of an indoor test session, at an average confidence of 0.78. Every live session is recorded, so we can replay it offline and tune detection without the camera attached.
The dashboard provides a human-review workflow that preserves the matching thermal image and bounding box for each detected person. Manual markers let operators supplement the model's observations.
We also built automated tests for critical interfaces, and offline map and terrain support.
What we learned
We learned that reliable detection depends heavily on data quality and evaluation design. Neighboring video frames can make results appear stronger than they are, so separating entire sequences and checking training ancestry matters. Training data also has to look like the real sensor: clean thermal datasets did not prepare the model for the Lepton's blur and noise.
We learned that integration is as important as detection: a correct detection paired with the wrong image, stale telemetry, or a flipped sign convention still produces an incorrect result.
Hardware testing exposed issues that documentation alone could not resolve, including sensor filtering and board orientation.
Finally, we learned that human review belongs in the design from the beginning. Source images, confidence scores, and clear indications of unavailable information help operators judge what the system actually knows.
What's next for VIGIL
Our next priority is completing hardware bring-up and controlled flight validation, progressing from stable manual flight to altitude hold, GPS navigation, and waypoint missions.
We need to calibrate camera mounting, heading, and image-to-telemetry latency, then measure location error against known ground targets.
For the AI, our live testing so far has been at close range. Next we want in-flight Lepton footage across different altitudes, backgrounds, and motion conditions. We'll measure missed detections, false-alert episodes, and detection delay alongside conventional accuracy metrics.
We also want systematic area-coverage routes, persistent review records, easier offline mission preparation, and feedback from search-and-rescue operators to guide the next version.
Built With
- c++
- esp32
- expresslrs
- inav
- platformio
- python
- react
- stm32
- typescript
- yolo
Log in or sign up for Devpost to join the conversation.