Vision is a very important part of life. Let's imagine that a normal person wakes up from sleep and suddenly lost his vision! It would be unbelievable for that person that he had lost his vision overnight. Common causes of sudden loss of vision include stroke or transient ischemic attack (TIA), acute glaucoma, retinal detachment, inflammation of the structures in the front of the eye between the cornea and the lens. Therefore, we, who have the power to perceive everything around us, can pretty well imagine that how horrific it would feel if one is visually impaired by birth or maybe a victim of sudden vision loss. An estimated 253 million people live with vision impairment, out of which 19 million are children who are below the age of 15. It is usually very difficult for visually impaired people to travel around in today's rapidly changing world. Current technologies in the market have failed to equip them in such a way that they can cross roads, avoid manholes and detect obstacles while travelling. They can’t always rely on human help. Without proper feedback from the surrounding, it becomes impossible for them to navigate and walking sticks are of little help in such situations. Apart from interacting with their environment, interacting with other people is another big challenge for the visually impaired. It is impossible for them to locate a known person in front of them. While communicating with other people it becomes very difficult for them to detect their age, gender, facial emotions, etc. There are some devices available which enable them to read books and road signs, but the experience is not as smooth as it should be.

Thus, we thought to make a difference and transforming the lives of the Visually Impaired people by building a solution which would revolutionize the way they interact with the world.

What it does?

The prototyped solution would have a life-changing impact in the lives of the Visually Impaired people. They would be able to experience their surroundings like never before through audio feedback based on their surroundings. Whenever they would like to understand or experience what kind of objects are present in front of them/ know their surroundings, captured frames using Image Captioning would be converted to a Voice based Assistant to give details of that situation in real time. With this product, they would be able to have precautions for dangers in front of them. They could detect obstacles while travelling, understand restaurants and hospitals nearby. Future developments include analysis of the text written in the book which they are holding and most importantly they would understand the facial expressions of the people they are interacting with. The visually impaired people would never require to rely on anyone’s help when they could have this fully assistive real-time solution for them. By utilizing this product the part of the brain which is used for creative and imagination skills would be enhanced which intern helps in the betterment of individuals.

How we built it?

The prototype will change into a wearable once we market it and make it scalable. This device is completely voice controlled. Using the latest advances in Machine Learning and Computer Vision we are trying to transform the lives of the visually impaired people. RICognEyes is capable of describing what’s in front of the person through the audio output. It can help in finding things around the user using state of the art object recognition techniques. It can tell the user the facial expressions, age and gender of the people in front of them immediately by harnessing the power of edge computing as most of the major computation will be done on the device itself.

Working: -Real-time Object Recognition using IMAGE CAPTIONING, coupled with sending that captured frame to the Oracle Cloud Infrastructure which when converts the captured image into text descriptions and after that converts the text to Sound Feedback by the assistant!

Challenges we ran into?

1)We were not that familiar with the QUALCOMM DRAGONBOARD, therefore we took time to study its data sheets and then start working slowly. It took us time to get the board entirely configured because there were Internet issues and a lot of updates had to be done on the board. Once that was accomplished, Cameras provided to us from the Hardware Lab were not compatible with DragonBoard 410C board, therefore we had to go out in the city and buy a USB powered Web camera. Lastly, Communicating between the Oracle Virtual Machine and the DraginBoard 410C was very very much hectic.

Accomplishments that we're proud of?

We knew it was a hard task to implement our idea on a totally new board and environments, but we challenged ourselves to fight till the end and we did it. Being able to transform our idea into a project that could be displayed feels great! We will never step back in life from trying something new every time. If we could stay up for 36 hours and accomplish our goal, we believe nothing is IMPOSSIBLE.

What we learned?

All of us in the team share the same vision, that giving back to society is everything. We believe that projects and prototypes should emerge out of Hackathons, Research Labs and be meshed with business models so as to solve real-time problems that our people face day to day. GIVING UP IS NEVER AN OPTION. EVEN IF WE FAIL, WE TAKE LESSONS FROM IT AND MEND THEM, PROCEED FURTHER AND FURTHER.

What's next for RICognEyes- Real-Time Image Cognitive Eyes?

We are really looking forward to the detection of facial expressions of people around a Visually Impaired person. Also, we want to create something that could scan whatever that is written in front of the person i.e MENU CARDS, POSTERS etc, that text would be read out loud in his ears with the help of the Voice Assistant.

Built With

Share this project: