BlindNav

Inspiration

As an electronic engineering student, I felt a sense of disappointment when I wasn't admitted into Computer Science courses due to the competitive public admission system in Malaysia, which favors bumiputera applicants. I found myself in Electronic Engineering instead, but my passion for software development remained strong. To fill this void and enhance my resume, I began participating in hackathons. After several unsuccessful attempts, I became determined to create something impactful.

The idea for BlindNav was inspired by two key moments: reading about an app for blind users to consume internet content and my realization of the broader challenges visually impaired individuals face in navigating their daily lives. This understanding sparked my commitment to develop BlindNav, an AI-powered tool designed specifically for the blind community to assist in navigation.

What it does

BlindNav is an AI-driven navigation app that provides real-time object detection and voice-guided assistance for visually impaired users. By utilizing advanced technologies like YOLOv10 for object tracking and OpenAI Whisper for speech recognition, BlindNav helps users understand their surroundings and navigate safely.

How I built it

BlindNav was developed using several key technologies:

YOLOv10 for Object Tracking: This AI model detects dynamic objects in the user's environment, providing essential context about moving and stationary items. OpenAI Whisper for Speech Recognition: This feature allows users to ask questions about their surroundings, converting spoken queries into text. OpenAI O1 for Answering: The app generates contextual answers based on detected objects, enhancing user understanding of their environment. Pyttsx3 for Speech Feedback: This tool converts text to speech, delivering real-time information back to users. The combination of these components creates a seamless experience for users, enabling them to navigate their surroundings more effectively.

Challenges I ran into

Throughout the development of BlindNav, I faced several challenges:

Balancing Low-Computational Needs with Accuracy: Ensuring that the app runs efficiently on mobile devices while maintaining high accuracy in object detection and speech recognition was a significant hurdle. Real-Time Processing: Achieving real-time feedback while keeping the system lightweight was critical to providing a positive user experience. Integrating APIs: Synchronizing the various APIs for object detection and voice interactions required careful planning and troubleshooting to ensure smooth functionality.

Accomplishments that we're proud of

I am proud of creating an innovative solution that addresses real-world challenges faced by the visually impaired community. Successfully implementing real-time object detection and speech recognition within a mobile application demonstrates both technical proficiency and a commitment to accessibility. Additionally, developing a user-friendly interface without relying on visual elements showcases my dedication to inclusivity.

What we learned

This project taught me valuable lessons beyond technical skills. I gained a deeper understanding of the importance of accessibility in technology and the impact it can have on improving lives. I also learned how to better empathize with the needs of visually impaired individuals, which inspired me to prioritize user-centric design. Furthermore, I enhanced my skills in AI and machine learning, full-stack development, and integrating various technologies.

What's next for BlindNav

Looking ahead, I plan to port BlindNav to Android to make it more accessible for real-world use. I aim to adopt a freemium model, with a free tier supported by ads and a premium tier offering advanced features. Eventually, I hope to integrate IoT microcontrollers to provide an even more seamless navigation experience, as relying solely on mobile devices for extended periods may not be ideal for blind users.