We often don’t realize how different the world is for disabled people, such as the blind and the deaf. To raise awareness, we wanted to create a technology that can both assist and empower them for a very long time. Since some of us have relatives who are blind, we decided to create an app that can take pictures or images, analyze what is in that photo, and read that response out loud. Vision-impaired people can thus use this app to scan their surroundings, read signs and papers without Braille, and freely explore their surroundings.
What it does
VisualEyes is a hybrid app that works on both web and mobile. A user can either take pictures themselves or upload photos on their camera gallery, and the app will immediately say out loud what is in that picture. It is able to detect people, objects, and scenery, making it robust and description. The user can also use VisualEyes to read street signs, store names, and handwriting, so they are able to read books and papers without relying on Braille. In addition, the website has a location heatmap feature which graphically shows where the user takes the most pictures. A history feature displays the most recent images.
How we built it
Challenges we ran into
Implementing the APIs was a challenge for us, especially when we were working with Hasura, because it was everyone’s first time using the API. It’s a relatively new technology and so we often had to ask mentors for help (shoutout to all the helpful mentors!). We also spent a lot of time trying to decide on what to do in the beginning of the hackathon deciding how we would turn our idea into a working prototype. So along with programming, we spent quite a bit of time doing independent research and utilizing the mentors to help us answer our questions.
Accomplishments that we’re proud of
We’re very proud of seeing how VisualEyes grew from a small idea to a full-fledged project. We’re also very proud of the fact that we learned how to use the Hasura API and Microsoft Cognitive Services API with the help of some amazing mentors, in such a short amount of time.
What we learned
We learned that while it is quite simple to come up with a list of ideas, it’s difficult to pick just one to commit to, especially if there was more than one idea that we really liked. We also learned some amazing new skills including how to use new technologies such as the Hasura API and the Google Maps API.
What’s next for VisualEyes:
We hope to keep refining the mobile side and making it more responsive. We would also want to incorporate camera functionality to be able to get descriptions of an environment in real time.