Sentinel

Inspiration

When disaster responders enter an unknown building, they are often making decisions with very little information: Is the path blocked? Is there a person inside? Is there fire or another hazard nearby? I wanted to build something that could go in first, take the risk instead of a human, and send back useful situational awareness in real time.

That idea became Recon Rover: a low-cost autonomous rescue rover that explores hazardous indoor spaces, builds a live map, detects possible victims, avoids obstacles, and streams everything to a browser dashboard. The goal was not just to make a remote-controlled car, but to give it enough autonomy to act like a first-look scout for emergency response.

What It Does

Recon Rover runs fully offline on the Arduino Uno Q. It combines camera vision, distance sensing, IMU data, motor control, and a web dashboard into one deployable system.

The rover can:

Explore an unknown indoor space autonomously
Avoid obstacles using distance sensors and a servo-mounted scan
Estimate its motion using visual odometry and gyro data
Build a live 2D occupancy map of explored space
Detect people using OpenCV MobileNet-SSD
Mark confirmed detections as possible victims on the map
Return roughly toward its starting point using breadcrumb navigation
Stream a live camera feed, robot status, map, and controls to a dashboard

At a high level, the robot follows a continuous loop:

$$ \text{sense} \rightarrow \text{localize} \rightarrow \text{map} \rightarrow \text{detect} \rightarrow \text{navigate} $$

That loop lets the rover keep moving, avoid obstacles, and update the rescuer’s view of the environment in real time.

How I Built It

I built the project as a hybrid robotics system with two main layers.

The microcontroller layer handles the hardware-facing work: motors, sensors, and servo control. The Arduino side reads the ToF sensor, ultrasonic sensor, IMU, and temperature sensor, then exposes simple bridge functions like read_sensors, set_motors, and set_servo.

The Python layer runs the autonomy stack. It uses OpenCV for camera input, visual odometry, and person detection. The rover estimates motion by tracking ORB features between frames and fusing that with gyro turn-rate data. Distance readings are ray-cast into a Bayesian occupancy grid, where each sensor update changes the belief that a cell is free or occupied:

$$ L_t = L_{t-1} + L_{sensor} $$

The dashboard is built with Flask and gives the operator a live camera view, occupancy map, robot heading, distance traveled, detection count, and control buttons for explore, return, and stop.

One important design choice was keeping everything edge-based. The model files are bundled locally, and the rover does not need internet access to run. That matters for disaster response, where connectivity may be unreliable.

Challenges

The biggest challenge was making computer vision work on limited hardware. Running visual odometry and object detection together is expensive, so I had to keep the camera resolution modest and only run detection every few frames.

Another challenge was localization drift. Visual odometry is useful, but it is not perfect. Blank floors, low-texture walls, and fast turns can reduce feature matches, so the map is an estimate rather than a perfect floor plan. I handled this by keeping runs short, adding breadcrumbs, and designing the return-to-base behavior as “roughly return home” instead of claiming centimeter-level navigation.

Hardware integration was also difficult. The motor driver, servo, ultrasonic sensor, ToF sensor, and IMU all had to work together without blocking the Arduino Bridge. I also had to adapt the navigation logic to the actual physical rover, including slower motor speeds, servo settling time, and the fact that the camera and ToF sensor pan together.

False positives in person detection were another issue. A single-frame detection should not immediately become a “victim” marker, so I added temporal confirmation: the same detection must persist across multiple frames before it is logged.

What We Learned

This project taught us that robotics is less about one perfect algorithm and more about making many imperfect systems cooperate. The camera drifts, sensors are noisy, motors behave differently under battery load, and real rooms are messy. The challenge is building a system that still behaves usefully despite those imperfections.

I also learned the importance of honest autonomy. Recon Rover does not pretend to solve full rescue robotics. Instead, it focuses on a practical and valuable first step: giving responders a safer way to see what is inside before they enter.

The most exciting part was seeing the full loop come together: the rover moves, senses, maps, detects, and reports back through the dashboard. That made it feel less like separate code modules and more like an actual rescue scout.