ECHO | Devpost

💡 Inspiration

ECHO was inspired by how much of everyday navigation depends on split-second visual cues. Especially in our information rich world where reading text, identifying objects, and understanding spatial layouts are almost assumed and something most people take for granted. A low branch, a misplaced chair, a crowded sidewalk—these are minor inconveniences for some, but real hazards for people with visual impairments. We wanted to explore how technology could bridge that gap in real time, giving visually impaired users more independence and confidence as they move through the world.

🔍 What it does

ECHO is a mobile app utilizing Apple iPhone Pro’s LiDAR technology to detect objects in the real world. ECHO will narrate what objects the user is approaching and alert when objects are very close to the user so that they are able to navigate complex spaces or avoid potentially hazardous obstacles. Our app also has a voice recognition feature where the user can prompt ECHO with commands like “Ok, ECHO, read out the text that is in front of me”.

⚙️ How we built it

We built this natively for IOS using React Native and Swift. Then we connected ElevenLabs for object narration and used the default Apple narration for the speech to text feature. Lastly, we used the built in AV foundation, and Google Cloud Vision API to identify objects with the iPhone camera. Frontend:
React Native: React Framework for Mobile Apps Tailwind: A customizable CSS utility framework for styling with classes. Swift: iOS native software language APIs: ElevenLabs: For custom voice text to speech narration Google Cloud Vision: For Object Detection

🪦 Challenges we ran into

ElevenLabs documentation felt unintuitive, so we would often use older versions of requests or use voices that don’t exist. We also experienced some issues recognizing voices for commands We tried to incorporate Gemma, but ElevenLabs is much better with text to speech or vice versa.

😁 Accomplishments that we're proud of

We were able to quickly set up the LiDAR for our app. We choose a clear voice that is easy for people to understand We are proud of our idea and the use cases since it can also help other people (i.e. construction workers) not just the visually impaired, especially with notifying users of obstacles and hazards

📖 What we learned

We learned that LiDAR can be accurate, but often fluctuates slightly on a distance AI’s will often use features that don’t exist or use wrong versions for libraries, so we need to audit changes if it involves libraries

🤔 What's next for ECHO

If we had more time some additional features or improvements we would implement: Personalized User Experience: AI algorithms can learn individual user behaviors, such as walking speed, preferred routes, and common destinations based on user favorites. This personalization results in tailored navigation, and helps build personalized mental maps that fit each user’s unique needs. Support Social Recognition: Facial recognition to help identify saved contacts or people previously met, letting the user know of their name when they enter a room, recognise social signals and gestures and facial expressions to help visually impaired users contribute to social interactions seamlessly.