With 1 million Canadians who are visually impaired and and 100,000 who are blind, the number is only predicted to increase in the coming years. However, the technology has not been following the demand for devices to assist those who cannot see. To make a product to change their lives for the better has been the inspiration. Bring back one of the six senses.
- Third-Eye captures images and recognizes things in them using the Watson Visual Recognition API.
- If there is a person within an image, facial recognition is performed to recognize any known people. For now, only faces of famous people that are contained in Watson's database can be recognized.
- For the list of things that are recognized within the images, the Watson Natural Language understanding API is used to find out which of these things are "entities". Entities are physical objects that the user would want to know about.
- Using a distance sensor, Third-Eye can detect how far these physical objects are from the user.
- Third-Eye tells the user using a text-to-speech application about the objects/people in front of him/her, and how far away it is. For people, Third-Eye says their names to the user.
How we built it
We use the Raspberry Pi, a Pi Camera module, a distance sensor, and wireless microphone for the purpose of feeding live video feed for computer vision analysis, object classification, distance/potential obstacle analysis, and facial recognition. Watson's Visual Recognition and Natural Language Understanding APIs were used.
Accomplishments that we are proud of
Even with many hardware drawbacks, we still accomplished most of what we wanted through all the late nights. We are so happy we could bring a product that could instantly improve the lives of so many people around the world.
What we learned
We learned how to use the Raspberry Pi Suite for the live video feed for computer vision analysis, object classification, distance/potential obstacle analysis, and facial recognition. We also learned how to work with Python on the servers and the cloud based processing in PHP as well as tools such as IBM Watson. Last but now least, teamwork!
What's next for Third Eye
We want to incorporate more learning algorithms and training to continue to improve recognition of objects and facial features of people.