3D reconstruciton of unexplored territroy with target

BlindSight

BlindSight is a small ground rover that drives itself into a space it has never seen, maps the space as it goes, and then drives back to where it started. No joystick, no pre-loaded floor plan, and no internet connection. You set it down, it figures out the rest.

Why we built it

There are places you don't want to send a person first: a building after an earthquake, a basement full of smoke, anywhere you'd rather risk a cheap robot than a life. Most "robots" at our level are really just RC cars with a camera bolted on. We wanted ours to actually make its own decisions about where to go, and more importantly, to be able to get itself back out.

What it does

You place the rover in an unknown room and it starts exploring. It keeps track of where it is, what's blocking it, and which directions still lead to space it hasn't seen, and it steers toward the unexplored areas. While it drives it's building a 3D map of the room in real time, marking out what's open, what's a wall, and what it hasn't reached yet.

When it's covered what it can reach, or when you tell it to, it plans a route straight back to its starting point over the map it just built and drives home. It doesn't just reverse its own path; it works out a direct route, the way you would if you knew the layout.

The whole time, you can pull up the map on your phone over the rover's own Wi-Fi and watch it fill in. You see the rover, the trail it's left, the walls it's found, where home is, and the path it's planning back. There's also an optional mode where it runs object detection and drops a pin on the map when it spots whatever you told it to look for.

How it works

The trick that made this feasible was not trying to do SLAM ourselves. An iPhone Pro already does extremely good camera + LiDAR + motion tracking for AR, so we let the phone do the hard localization work and stream its pose and depth data to the rover. The rover's brain treats the phone as its eyes and inner ear.

From there the brain (Python, running on an Arduino UNO Q) takes the phone's depth data, turns it into a map of free and blocked space, runs frontier exploration to pick where to go next, and uses A* to plan paths around obstacles and back home. A small web server pushes the map out to the phone over a websocket. A separate motor controller on the car takes simple drive commands over serial and runs the wheels.

One thing that saved us a lot of pain: we wrote down the exact format of every message that crosses between the brain, the firmware, and the phone viewer before we wrote much code, so three of us could build three different parts at once without constantly breaking each other's stuff.

We also started on a separate, offline pipeline that uses the same phone LiDAR and camera to build a full photorealistic 3D model of a room after the fact, including a small tool that checks the captured data is correctly scaled before you spend GPU time on it.

What gave us trouble

Getting clean data off the phone was harder than any of the actual robotics. The USB streaming would die intermittently, the phone would overheat and quietly shut its camera off, and the gap between "the laptop can see the phone" and "the app is actually sending frames" ate a real chunk of our time.

The other gap is the one every robotics project hits: the navigation logic works perfectly in simulation, but the physical car needs careful calibration before it behaves the same way in a real room. Telling it to "go straight at 0.2 m/s" only means something once you've tuned the motors, and that's the difference between a clean exploration run and driving into a wall.

Where it stands and what's next

The decision-making is solid and tested: the mapping, the exploration, the path planning, and the return-home logic all work and are verified in simulation. The next step is hardening it on the physical car: calibrating the motors, adding safety stops, and proving that "return home" actually parks the rover back on its starting spot, even when home is around a corner and out of sight. After that, we want to ship the offline 3D reconstruction so every mission leaves behind a full 3D model of the space the rover explored.