How it works
Imagine for a moment that when you read, it looks like this. link For those with dyslexia, they don't need to imagine anything. This is what they struggle with every single day.
Friends and family who struggle with dyslexia inspired us to create a mobile app that makes it easier for dyslexics to read text in their daily environment.
What it does
Our app is called DyslexiLens, and it scans text from the camera's video feed in real time, edits it, and redisplays the dyslexia formatted version on the screen. It's quick, extremely simple to use, and makes it easier to read tough sentences or words on the fly.
How we built it
Optical character recognition and augmented reality are more accessible to developers than ever before with a number of open source SDKs available to them. We've seen augmented reality used for games like PokemonGo, and optical character recognition for text to speech and other accessibility related apps. To take things a step further, we chose to combine these technologies and explore a new type of learning tool.
Challenges we ran into
Using an API to perform OCR on the video frames was too slow, so we chose to use an open SDK that would perform the processing on the client. We used the community edition of the Anyline SDK which had limitations (such as a watermark that we cannot place a view over). There is no open SDK that renders text seamlessy within an image, so we had to implement our own solution which is less than ideal and needs improvement.
Accomplishments that we're proud of
We achieved our proof of concept, and came across various libraries that could be combined (with much more development time) into an image rendering solution that will meet our needs.
What we learned
Transposing text onto video frame that are constantly moving is not trivial. There are varying levels of dyslexia and there isn't a silver bullet solution. Changing the contrast and format of text isn't enough to create an immersive reading experience for the user.
What's next for DyslexiLens
Continue developing our app after hack the north and make it available on both the apple and google play store.