SparkyScout

Inspiration

SFU's Academic Quadrangle is a maze. On the first day of term, new students walk in circles looking for rooms like AQ 3046, and anyone who can't easily read a map or a sign has it worst. We wanted a guide that doesn't just show you the way but walks it with you.

What it does

Sparky Scout works like calling a ride, except the car walks you to your classroom door.

Pick a room: search any room, or tap "Find the nearest..." for a washroom, water fountain, computer lab, elevator, stairs, vending machine or exit, each with a walking time.
Request a Scout: the app shows the route, the walking time and distance, and a step-free (elevator only) option. The nearest free Scout drives over and pulls up next to you.
Follow: the car leads at walking pace, and the app shows the distance left on the level and the car's battery.
Change levels: at stairs, Scout hands off ("Take Central stairs to Level 10000") and your route carries on from there on the map.
Arrive: you get the walk time and distance, and Scout heads back to its dock.
Talk to it: "Chat with Scout" is powered by ElevenLabs. Ask out loud ("Where is the nearest washroom?") and Scout answers by voice, with a button to go there. With no phone at all, press the button on the car and say the room.

Any of the 1,923 rooms in AQ, WMC and MBC can be picked as a destination, and the app now covers the Applied Sciences Building (ASB) too.

How we built it

Voice (ElevenLabs): ElevenLabs Scribe turns speech into text. We prime it with every campus building and room name as key terms, so "A Q thirty forty six" comes back as AQ 3046. The car reads the room back ("AQ 3046, on the 3000 level of the Academic Quadrangle") and only drives on a yes. Every instruction is spoken in an ElevenLabs Flash v2.5 voice, with Piper as an offline fallback. Gemini picks the room and handles confirmations through three tools.
Hardware: a stock RC car (75 x 41 cm, four-wheel drive) that we never opened. An ESP32 stands in for the remote's steering wheel and trigger, feeding 3.3 V into the potentiometer wires, so the remote thinks a hand is holding it. A laptop rides on the car with two webcams as a 10 cm stereo pair, an iPhone as the IMU and GPS, a mic, a speaker and a push-to-talk button.
Maps: SFU's public GIS server gave us 2,441 room polygons. Our pipeline turns them into Nav2 floor grids and a campus route graph with 5,518 nodes.
Navigation and vision: ROS 2 Jazzy with Nav2 (a Hybrid-A* planner, because a car can't turn on the spot), RTAB-Map stereo odometry fused with the phone IMU, YOLO11 for people, SegFormer for floor edges and PaddleOCR for room signs.
App: a phone web app (FastAPI and rosbridge) to search rooms, see the route on the floor plan, and tap "I'm holding the door".

Challenges we ran into

The car can't turn on the spot. Every planner and recovery behaviour assumed a robot that can spin. We rebuilt them for car steering. When the car is lost, it now asks you for the nearest door number instead of spinning to look for a sign.
No wheel encoders. Nothing on the car measures distance, so the car stops the moment vision loses track rather than driving blind.
Maps vs. reality. Wall lines in the GIS vanished at coarse resolution, so one route went through a wall. Door nodes sat inside the car's footprint, so the car stalled short of the door. We found both in simulation and fixed them.
Driving a car through its own remote. The ESP32 dies above 3.3 V, and the remote's pots can run higher, so we had to measure every voltage before wiring anything.

Accomplishments that we're proud of

In simulation, the car reached AQ 3150 at the end of a 275 m, three-turn route and stopped 0.56 m from the door.
A simulated elevator trip from AQ level 3 to level 6 relocalized on the new floor and arrived 0.53 m from the door.
A full voice loop with no screen: ElevenLabs hears the room, reads it back, and speaks every turn.
184 automated tests, and a hardware-in-the-loop simulator that runs our real ESP32 firmware.

What we learned

Voice is the most accessible interface for a robot that moves: you can't read a screen while you're following one.
Telling the model the exact room names to expect makes speech-to-text far more reliable than general correction afterwards.
Simulating early caught bugs that would have crashed the real car.