BlindSight

Haptic Belt
Push to Talk, photo identification helper
Demo mode: haptic belt input and intensity
Demo mode: depth perception view

Inspiration

Over 35% of people will experience lifestyle-altering visual impairment in their lifetime, yet assistive tools remain expensive and difficult to navigate. One of our members' grandfathers has been blind for over 15 years, only being able to navigate life with the assistance of his also aging grandmother. So we designed BlindSight, an assistive tactical device that recovers day to day autonomy and is comprised of inexpensive technology and an app on your phone. It uses the LiDAR scanner and RGB camera from your phone to process the surroundings and determine which haptic motors to turn on and with what intensity.

What it does

Keeps you safe! Detects obstacles, producing dynamic haptic feedback, increasing intensity as you move closer to the obstacle
Have real conversations in real time! Speak aloud and get an audio response
Gain clarity, identify and describe where you are

How we built it

ElevenLabs (STT and TTS APIs)
Gemini API (responding to the request/question)
Swift (developing the mobile application)
iPhone Pro LiDAR Scanner
RGB data from the main iPhone camera
Microphone input from the iPhone
iPhone Speaker output for TTS
DC motors controlled by a L298N motor driver to simulate vibration
ESP32 to receive instructions to spin the motors from the iPhone

Challenges we ran into

We faced A LOT of hardware challenges;

Missing parts in orders we picked up
Waiting for over 2 hours in line for a depth perception camera we didn't end up getting
Stripping and soldering used wires last minute, because they ran out.

By the time we had an idea and the hardware to back it up, it was already 6pm on Saturday. So we had to LOCK IN hard the rest of the day.

Accomplishments that we're proud of

Pivot and adapting despite the hardware limitations. Unlike many other teams, we met and created the team on the day HTN started, but surprisingly we hit it off and were able to collaborate and pull through by tweaking and constantly adjusting our ideas.