Inspiration

One of our close relatives is blind, and moving through unfamiliar places is hard for them. Obstacles like chairs, bags, and people can appear without warning, and they often have trouble seeing what is in front of them. Watching them get around made us realize how much a sighted person takes for granted: seeing what is ahead and knowing which way is safe.

That is what pushed us toward Brim. We wanted to give a blind person a second set of eyes that speaks up early. The camera from QNX showed us how: a webcam on a hat, running on a Raspberry Pi 5 with QNX, can watch the path ahead and turn what it sees into short spoken warnings, all on the device.

We also wanted to focus on the things that are not expected. Blind people already know their routines well: cooking, getting ready, etc. The hard moments are the surprises, like a chair pulled out into a walkway, a bag left on the floor, or someone walking in front of them. Those are the moments we wanted to build Brim for.

What it does

Brim is a hat for blind and low-vision wearers that uses a small camera and an on-device AI model to help them move around safely. A Raspberry Pi 5 running QNX reads a camera mounted at head height, finds people, chairs, tables, bags, and other obstacles, and works out whether the path ahead is clear. When it isn't, Brim tells the wearer which way to step, or to stop if there's no clear side. Speech goes to a companion phone app, since the Pi has no earpiece, and safety alerts take priority over everything else.

How we built it

Challenges we ran into

This was our first time working with hardware, and that made the project much harder than we expected. We planned around a distance sensor, but we didn't have one, and we had no other sensors to measure how far away obstacles were. We had to identify our constraints early and design around them. The camera became our only source of information, and Brim estimates how close something is from how large it appears in the frame. Since the Pi has no earpiece, we also had to send speech to a phone app.

The hardest problem was speed. Brim needs to warn a blind person right away, but the open-source EfficientDet-Lite0 model we were given took about 74 ms per frame on the Pi, and the camera ran at only about 15 frames per second. That meant warnings could lag behind what the person was walking toward. We fixed it by running the model on every other frame instead of analyzing all of them, so Brim could get back to the user sooner. We worked through each change one at a time, and in the end we found an approach that works and still gives people real help getting around.

Accomplishments that we're proud of

We're proud that we got Brim working and built our first hardware project. We wanted to build something accessible, something that helps people who can't see and who most tech isn't designed for. We knew it would be a challenge, but we're happy we came out with a working result that people can use in their everyday lives. Working with the Raspberry Pi and QNX was also a great learning experience, and getting the hardware to work was one of the most rewarding parts of the project.

What we learned

What's next for Brim

What's next for Brim is that we want Brim to speak more languages so more people can use it, and to detect more of what's around them, including hazards on the floor. The biggest hardware change would be adding real distance sensors, which we didn't have this time, so Brim wouldn't have to estimate distance from the camera alone. We also want to move the hat into glasses, like Meta glasses, so it's easier to wear all day and can give instructions without a phone in hand. Beyond navigation, we want Brim to help with everyday life, such as cooking and getting ready, by learning new patterns and adapting to each person's routine and lifestyle.

Built With

Share this project:

Updates

Submission history