Read Now

Inspiration

As a child, reading more advanced books and articles beyond my reading level, I would be frustrated that it was dense and not easily understandable. We created a tool to help children learn to read, and help them access knowledge.

What it does

User takes a photo of printed text with their camera, and the app processes this text and displays it on the screen. The user can have the text read out to them, and the current word will be bolded.

How we built it

We used Microsoft Computer Vision API to get the text from the photo. Then Android Text-to-Speech to read the text aloud. We then created a simple GUI to display the text and highlight words as the app spoke. We also wrote code to identify the hardest words in the given picture (by building a custom data-set based on the average age someone learns the word) and then provide definitions for those words.

Challenges we ran into

We initially set up the Microsoft Computer Vision API. None of us had ever worked with this API before or images in Android so figuring out how to pass the file was more involved than we anticipated it to be. After parsing the JSON to get the words from the picture, and using the Android Text-to-Speech API, we wanted to display the words, and highlight, the word that was currently being said. Synchronizing the speech and the highlighting was difficult because the API doesn't notify you word by word, so we synchronized at the end of sentences instead.

Accomplishments that we're proud of

We figured out the HTTPPost to use the Computer Vision API. We also did research on what makes a word difficult, and created a custom data-set.

What we learned

3 of learned how to make Android apps in Android Studio. We also realized that a large number of useful APIs are accessible online, and that using these allows you to create something much more complex and sophisticated than you would have been able to otherwise.

What's next for Read Now

We want to make the app remember pages, allow users to input multiple pages at a time, and learn the user's reading age to better determine difficult words for them. We would also like to compile a set of child friendly definitions.