BlindSpot - Hack the North

The Blindspot Camera module

Inspiration

Millions of people live with visual impairment to a certain degree. The cane has been the primary mobility aid for over a century, and it's a brilliant invention, but lacks in reach and tells you nothing about what is there. We wanted to build something that gives back a sense of the space and provides a layer of awareness. So impaired people can really sense their surroundings.

What it does

A camera that can be clipped to the collar of your shirt splits the view into 6 quadrants (top left, top center, top right, bottom left, bottom center, and bottom right).
As obstacles get closer, a vibration pad on that side vibrates more frequently so you can get a better sense of where things are.
Additionally, utilizing LLMs like Google Gemini alongside the Elevenlabs API we're able to provide users with audio feedback from their phones about their surroundings, helping users reclaim some form of object recognition by using AI to tell them what's around in their field of view, this also gives them access to the AI as a chatbot/ smart assistant answering questions about the world around them. ## How we built it Blind spot is composed of:
A raspberry pi 3v for majority of the onboard processing
A raspberry pi camera to provide the user with a field of view
2 L298N motor driver modules to control and power the motors we used
4 DC motors with small lopsided weights to make makeshift vibration motors that sit on top of the straps of the satchel we used
A 5V power bank to power the raspberry pi 3v
a 9V battery and a 7.4V LiPo battery to power the motors We used Python and OpenCV to compare each frame to the last one to tell how close things are, For the voice part, we send a photo to an AI model and use ElevenLabs to speak the answer. We 3D printed the cases. The vibrating motors run on the Pi itself (it can compute on its own) so it's instant. The AI voice only runs when you ask.

Challenges we ran into

We killed a Raspberry Pi and a motor driver due to a wiring mistakes or deffective products, but despite this getting a single camera to sense "how close" something is was hard, since one camera can't really see depth. The motors were janky and alot of the 3d prints that were created, weren't flexible enough for some of the parts. Google also broke their own API keys mid-event, which cost us a lot of time.And getting the phone to talk to the Pi over the venue WiFi was a pain, since public networks often block devices from reaching each other.

Accomplishments that we're proud of

We turned a cheap camera into a device a blindfolded person can actually walk around with and be able to sense their environment. The buzzing tells you where something is and how close it is. On top of that, the AI can look at a scene and describe it, warn you about dangers, read text out loud, and tell you what you're holding. Best of all, we kept it working. After every dead board and broken part, we still ended up making something cool by using an 11-year-old dev board.

What we learned

Throughout this project we worked heavily with raspberry pis a control board that none of us had much experience using, so figuring out how to boot the operating system we needed

What's next for BlindSpot

While we do feel as though we succeeded in our goal of making a device that increases the spacial awareness of people with visual impairments, we also understand that factors like having limited resources and the time restraints of the hackathon had a large impact on the quality of the final prototype that we were able to build here. So some ideas that we would like to look into for future designs are: