Inspiration
Sight is so automatic we forget it's doing anything. Every step you take, your brain is constantly mapping the world around youm; that doorframe, that chair leg, that person walking towards you. For 43 million people that's just gone. I wanted to build something that gives it back. Not a white cane. Not a guide dog. A wearable AI that actually understands what it's looking at and talks you through it in real time.
What it does
EchoNav is a wearable navigation assistant for the visually impaired. A RealSense D455 depth camera is mounted on the chest and streams data to a laptop in a backpack. Two parallel loops run continuously: a fast loop that processes depth frames at 2Hz to detect obstacles in left, center, and right zones with exact distances, and a slow loop that sends RGB frames to Claude's vision API every 3 seconds which produces vivid, detailed descriptions of exactly what the camera sees. A vending machine with spiral dispensers. Exposed ductwork across a dark ceiling. The underside of a chair with metal springs. Both outputs are fed into a priority TTS queue and spoken through Bluetooth earbuds. Obstacle alerts always cut in ahead of scene descriptions so critical warnings are never delayed.
How I built it
The entire pipeline runs on ROS2 Jazzy. The RealSense ROS2 driver streams depth and RGB topics which are consumed by two custom Python nodes. The obstacle detector uses NumPy to divide each depth frame into zones and finds the 5th percentile distance in each to filter noise. The scene describer encodes RGB frames as base64 JPEG and calls the Anthropic API. A third node manages the TTS queue using festival for audio output. The chest mount was 3D printed in PLA with a 30° downward tilt and strap attachment. Everything runs off a LiPo pack in the backpack.
Challenges I ran into
As a solo builder I had to design every single aspect from scratch; mechanical, software, and embedded systems simultaneously with no one to hand anything off to. The Raspberry Pi provided was substantially underpowered to run the RealSense camera, so I had to pivot mid-build and redesign the architecture to run everything on a laptop in my backpack. The 3D printer broke on the last print. And I spent my last $5 on API tokens to get it over the line.
Accomplishments that I'm proud of
Getting a fully wearable, end-to-end system working in under 24 hours as a solo builder. The dual-loop architecture works exactly as designed where obstacle alerts fire in real time while scene descriptions fill the gaps. Doing a blindfolded walk-through of a room with obstacles and navigating successfully using only audio is something I'm genuinely proud of.
What I learned
ROS2 is incredibly powerful but and had no idea it could be used for things other than autonomous robots. Depth cameras produce a lot of noise and naive minimum-distance approaches don't work — percentile-based filtering makes a huge difference. Simplicity wins under time pressure, ditching the Pi and running everything on one machine saved hours.
What's next for EchoNav
Haptic feedback motors in the pockets that vibrate with intensity proportional to obstacle distance;left motor for left zone, right for right. A fine-tuned vision model specifically trained on navigation-relevant scene description. A much smaller custom enclosure so the whole system fits in a slim vest rather than a backpack. A more natural, human-sounding TTS voice so the experience feels less robotic and more like a calm guide walking beside you.
Log in or sign up for Devpost to join the conversation.