An estimated 253 million people live with vision impairment, and 217 million have moderate to severe vision impairment. This affects these people with their daily quality of life. Our inspiration was drawn directly from a teammate's personal story about encountering a child who struggled to learn piano because of his vision impairment. With EyeDentify, he will be able to overcome his challenge with ease, allowing him to engage in his daily activities like his peers.
What it does
EyeDentify is a real-time, web application that uses "smart" zooming to enlarge text. When pointed at some text, it automatically detects the text, enlarges it and displays a high contrast text overlay. This allows for low vision users to be able to automatically magnify text and be able to read without struggle.
We specifically designed a succinct user interface for low vision users to use. Users can use the tool without clicking a single button. Arrows shown on the interface also guides people to find related text fields.
The high contrast text also helps people with color blindness be able to view text that otherwise would be impossible to read.
How we built it
It's built using advanced web technologies. We use WebRTC to enable any smart phone to be able to run the app. It captures the camera stream and does real time processing on it. We compute optical flow for image stabilization and use Microsoft Cognitive Systems OCR api for feature and text detection.
Challenges we ran into
- Defining canvas when zooming in and zooming out.
- Capturing texts in clusters and determine the location of texts to draw out guiding arrows.
- Resizing texts after zooming in to render easy to read text boxes.
Accomplishments that we're proud of
- Hacking a project that cultivates equity and access for low vision people, enabling them to see the world in a different angle.
- Building a web app that seamlessly integrates into various technology that people can easily get access to.
- Integrating Microsoft Cognitive Services API OCR technology to enable text recognition.
What we learned
- Working as a team and never give up when running into challenges!
What's next for EyeDentify
We had a bunch of features that weren't quite polished enough to demo, but we'd really like work more on them:
- Streaming to an external monitor for even larger magnification.
- Saving images to the device for later perusal. This would be useful for "realtime" situations such as lectures on the board.
- Integrating handwriting detection
- Detecting and magnifying other points of interest