Blindspot

Life-saving, image-recognition powered glasses for the blind with distance sensors and an emergency call system that, through audio speech output, let blind users navigate the world safely.

Comment 9

Inspiration

As a team of engineers, we were inspired by the challenges faced by the visually impaired. We recognized the need for a technology that could empower the blind to navigate their surroundings safely and independently. We have also all been fascinated with the recent advancements of machine learning and artificial intelligence and their real-world applications. Thus, we set out to create a machine-learning solution that could make a real difference in the lives of those with visual impairments.

What it does

Blindspot is a prototype of image-recognition glasses designed for the visually impaired. The glasses use OpenCV to scan the user's surroundings, identify objects and people, and announce them aloud along with the distance in centimeters. Familiar faces can be registered for immediate recognition, and an emergency video call function through Twilio allows the user to share their live feed with a contact in case of an emergency.

How we built it

Using OpenCV and Twilio, we built a prototype of image-recognition glasses that featured a webcam and distance sensor. The glasses were able to identify and announce objects and people within the user's vicinity, including the distance in centimeters. We also implemented a feature to register familiar faces, allowing the glasses to announce them by name upon detection. Finally, we added an emergency video call function through the services provided by Twilio Video Room Library to enable the user to share their live feed with a contact in case of an emergency. This emergency feature relies on a Selenium-integrated Python program. The program can automatically navigate to our web conference page and join a room with the proper computer sensors enabled and communicate with emergency guidance.

Challenges we ran into

One of the biggest challenges we faced was optimizing the technology for real-time use. Object detection algorithms in OpenCV often involve a trade-off between accuracy and efficiency. More accurate algorithms usually require more computational resources and are slower, while faster algorithms may sacrifice some accuracy. We needed to carefully select the appropriate object detection algorithm and fine-tune its parameters including the confidence intervals to strike a balance between accuracy and efficiency. Another difficulty we faced was connecting the actual Aurdino to the OpenCV. We needed to ensure low-latency performance when connecting Arduino to OpenCV. Data transfer and processing delays impact the real-time performance of our system, which led to delayed results. We carefully optimized the data transfer and processing pipeline by testing out a myriad of algorithms to minimize latency and ensure real-time performance

Accomplishments that we're proud of

We're proud to have developed a technology that has the potential to make a real difference in the lives of the visually impaired. Our image-recognition glasses provide a new level of independence and safety for blind individuals, empowering them to navigate their surroundings with confidence. We're particularly proud of the real-time capabilities of our technology, as well as the inclusion of an emergency video call function to provide additional support in case of emergencies. The video call system is a web app that we implemented using the services provided by Twilio Video Rooms. A trigger by the customer automates the entire process of the video call and the emergency "family member" will be able to access the view of our customer and guide him/her through any emergency situation.

What we learned

During our research, we discovered that many of the existing assistive technologies for the visually impaired were limited in their capabilities. We used the pre-trained version of the YOLO(You Only Look ONCE) deep learning model to detect people and objects. During training, the model learns to detect objects based on the annotated data, and the parameters of the model are updated iteratively to minimize detection errors. We also learned and improved our video call web application by adding the Selenium integrations into our Python program that can automatically navigate to our web conference page and join the video room.

What's next for Blindspot

Moving forward, we plan to continue refining the technology and expanding its capabilities. We aim to optimize the accuracy of the distance measurements and the object and face recognition capabilities. We also plan to explore additional features, such as a text-to-speech function for reading signs or written materials aloud, and what we find the most exciting, automating transactions by identifying items as "buyable" and even being able to make transactions by just looking at QR codes. We're even planning on making this compatible with the blockchain and cryptocurrencies. Ultimately, we hope to bring our image-recognition glasses to market and make a positive impact on the lives of those with visual impairments.

+ 25 more
Share this project:

Updates