Rescue Scout

A low-cost robot that searches a building with no map and tells firefighters where the casualty is, before anyone goes in.

Comment

Inspiration

Firefighters often have to search a smoke-filled or unsafe room before they know what's inside. Robots that can go first exist, but these can cost up to £160,000, and they still need a person driving them. We wanted a scout that's cheap enough to lose, small enough to get under furniture, and drives itself.

What it does

Rescue Scout is a small RC car with an FPV camera. You tell it what to look for and it searches a room it has never seen, with no map. Each step it looks, decides, makes a short move and looks again. If the target isn't in view, the car stays still and pans its camera left and right to look around, then turns towards whatever it spotted. When it finds the casualty it drives up, stops about half a metre away and reports where they are relative to the entry, a photo, and the route it took. One key press stops the car and hands control to a human.

For testing we used a small toy as the "casualty", so we could run it again and again safely.

How we built it

  • Eyes: NVIDIA Nemotron 3.5 Super VL on Nebius Token Factory reads each camera frame and returns a JSON decision: is the target visible, where is it across the image, where does it touch the floor, is anything close, which way to move.
  • Planner: Nemotron 3.5 Lightning reads the robot's log every few steps and picks where to search next. The log is the map.
  • Look-around: when the target is out of view, the camera pans left and right and Nemotron checks each view before the car moves at all.
  • The model says where, our code decides how far: Nemotron reports where things sit in the image (0–1000 scale). A ground-plane calibration from ruler photos turns that into centimetres, and our code steers towards the target, creeps in on the final approach and stops at a set distance.
  • Reflexes: our own code blocks forward moves into anything within 45 cm, detects when the car is stuck, and the car's ESP32 stops the motor if the laptop goes quiet for 350 ms. Optional rear ultrasonic sensor stops it reversing into things.
  • Hardware: toy drift car, ESP32-S3 driving the steering, motor and a pan/tilt camera mount over Bluetooth LE, Walksnail Avatar HD camera streaming to a laptop through a USB capture card. About £470 for the prototype; AI cost about 1–2p per search.
  • Dashboard: a Flask page showing the live frame, the model's decision, the search plan and the final report.

Challenges we ran into

  • The models we planned to use (Nemotron Nano Omni, Cosmos) were Dedicated-only on our account, so we benchmarked and switched to Nemotron 3.5 Super VL.
  • Every model called almost everything an obstacle until we defined "close" as within 30 cm.
  • Decisions took about 4 seconds. Shorter prompts, smaller images and turning reasoning off brought that to about 1 second.
  • Vision models answer image positions on a 0–1000 scale, which broke our first distance readings until we caught it. Moving to the car's own wide-angle camera meant recalibrating from scratch.
  • Real hardware fought back: a cheap speed controller that brakes before it will reverse, steering limits that only allowed half a turn one way, and a battery mounted off-centre that pulled the car sideways. Each one showed up in the robot's logs before we could see it.

Accomplishments that we're proud of

  • We benchmarked Nemotron 3.5 Super VL against NVIDIA Cosmos 3 Super Reasoner and two other open vision models on 34 floor-level photos. Nemotron and Cosmos both found the target in 12 of 13 photos with zero false alarms in 21, but Nemotron located it about 8x more precisely (1.7% vs 13.6% position error), faster (1.1 s vs 1.5 s), on a public pay-per-use endpoint.
  • Single-camera distance within 3 cm up to 1.2 m on the car's own camera, the range where stop and turn decisions happen.
  • A working end-to-end search on a real car: found the target in 5 of 5 runs when it was ahead or to the side (3–4 decisions, about 10–15 seconds per run), and found it from behind by looking around with the camera first. So far we have tested it in one room, so wider testing is the next step.

What we learned

Vision-language models are good at where something is in an image and poor at how far away it is, so combining them with simple geometry beats asking for distances. Letting the model decide what and our code decide how much made the car far more reliable. We also learned that speed comes mostly from asking for less output, not from a smaller model.

What's next for Rescue Scout

  • A thermal camera, since a normal camera can't see through thick smoke
  • Continuous tracking between Nemotron calls, so the car steers every frame and Nemotron decides every second
  • Side and rear distance sensors for corridors and multi-room search
  • A cheaper build (ESP32-CAM over Wi-Fi instead of an FPV kit), aiming under £100
  • Several scouts searching one building and sharing a map

Built With

Share this project:

Updates

Submission history