Inspiration
To give artificial vision to the visually impaired people.
What it does
It assists them to make their own meal
How we built it
The easiest way to access the camera and the microphone was to use a smartphone device. We made an android application that acts as an assistive device for the visually impaired by mutual communication between the application and the human. The subject tends to ask queries like "am i near the oven?" and the smartphone device replies by giving out an answer "yes you are near the oven.what would you like to do ?" or "no you are not near the oven.", accordingly. If the user has to start the oven, the assistant guides him/her to the button on the oven. we used google OCR, to find the text from the video feed of mobile camera and detect the start button for helping the subject with the navigation. to the button.
Challenges we ran into
The initial problem we faced was with the lack of availability of GPU which made it difficult for us to differentiate between the oven and the stove.
Accomplishments that we're proud of
we were able to parse user quires and help him with the navigation for cooking his own meal with a proper closed loop system.
What we learned
What's next for Hestia
Built With
- google-web-speech-api
- mobile-net
- ocr
- tts
Log in or sign up for Devpost to join the conversation.