AEye | Devpost

Inspiration

Worldwide, over 2.2 billion people live with vision impairment, facing daily challenges that many of us take for granted—navigating streets safely, shopping independently, and handling cash transactions. We were inspired to create a solution that could empower blind and visually impaired individuals to move through the world with greater confidence and autonomy. Our goal was to leverage cutting-edge AI technology to become a trustworthy companion that provides real-time assistance for everyday tasks.

What it does

Our app transforms a smartphone into an intelligent navigation and assistance tool for blind users through three core features: Smart Navigation: As users walk, the app continuously analyzes their surroundings through the phone's camera and provides real-time voice guidance about what they're approaching—obstacles, crosswalks, storefronts, and other important landmarks—helping them commute safely and independently. Intelligent Shopping Assistant: When grocery shopping, users can point their camera at products, and the app reads packaging information, checks ingredients against their dietary restrictions and allergies, and helps them make informed purchasing decisions through conversational voice interaction. Currency Counter: The app accurately identifies and counts cash currency, enabling users to make cash transactions confidently without fear of being shortchanged or cheated.

How we built it

We built the application using Gemini's vision capabilities to process camera feeds and understand the environment in real-time. The visual data is analyzed to identify objects, read text, and interpret scenes. ElevenLabs text-to-speech technology converts all information into natural-sounding voice output, creating a seamless auditory interface. We focused on creating an intuitive voice-first experience that requires minimal interaction, allowing users to receive continuous guidance while keeping their hands free.

Challenges we ran into

Latency was a massive problem for our project. We wanted to make sure users received information quickly enough to be actionable—especially for navigation where split-second awareness matters. Balancing the quality of AI analysis with response speed required extensive optimization. We also struggled with handling varying lighting conditions and camera angles, ensuring the app could work reliably whether indoors or outdoors, in bright sunlight or dim stores. Creating natural conversational flows for the shopping assistant was another challenge, as we needed to present information concisely without overwhelming users.

Accomplishments that we're proud of

We're incredibly proud of creating a functional multi-feature accessibility tool in such a short time. Successfully integrating real-time computer vision with natural voice interaction is something we're particularly excited about. The currency counting feature achieved impressive accuracy across different denominations and conditions. Most importantly, we built something that could genuinely improve people's daily lives and foster greater independence for the visually impaired community.

What we learned

This project taught us immensely about accessible design—that true accessibility means designing for voice-first experiences, not just adding audio to visual interfaces. We learned how to optimize AI model performance under tight latency constraints and gained valuable experience working with vision AI and speech synthesis APIs. We also developed a deeper appreciation for the challenges faced by blind and visually impaired individuals and the importance of building technology that's inclusive from the ground up.

What's next for "A-eye"

We plan to expand the navigation feature with GPS integration for turn-by-turn directions and crowd-sourced accessibility information about businesses and public spaces. We want to add support for multiple currencies and languages to make the app globally accessible. Future versions could include object finding capabilities ("Where did I put my keys?"), social features to describe people and social situations, and integration with smart home devices. We're also committed to working directly with the blind community to refine features based on real-world feedback and ensure we're building something that truly meets their needs.