BeyondWords

BeyondWords tackles Dyslexia with Artificial Intelligence. It serves as an AI Reading Companion for individuals of all ages with Dyslexia, enhancing their reading comprehension and fluency.

Comment

Inspiration

Dyslexia is a major cause for limited reading proficiency around the world. BeyondWords empowers individuals with dyslexia to read and comprehend any text with the help of Artificial Intelligence.

There are more than 780 million individuals around the world of all ages who are experiencing dyslexia, and 40 million in the United States, and only around 2% of them receive any diagnosis. What is really unfortunate is that the high school dropout rate for children with reading problems is a whopping 62%, with the main factor being reading disabilities like dyslexia. This is what devastated me the most. Just because of the fact that they interpret text differently, they were not able to perform as well as they desired, leading to them having to drop out of school. Knowledge is vital, and I believe everyone should have the ability to read whatever they would like to and are curious about without any hassle. This is why I created BeyondWords, with a goal to help tackle the challenges faced by individuals to Dyslexia, increasing their reading comprehension and fluency overall.

What it does & How I built it & How it works

BeyondWords is an Artificial Intelligence Reading Companion for individuals experiencing Dyslexia. People with dyslexia find it very difficult to read and interpret words and characters. Apart from text that is modified specifically for them, almost every other text around them (books, bills, brochures, and even anything with small text like ingredients printed on food items that increases the reading difficulty) is almost impossible for them to read and understand. BeyondWords tackles this challenge by providing several dyslexic-friendly features powered by Artificial Intelligence to help them interpret the text better.

Initially, the user takes a picture of the text that they have trouble reading. Only the text content from the image is extracted from the image using Google Machine Learning Kit's Optical Character Recognition (OCR) Library.

The extracted text using OCR is then converted and displayed in a special text format designed specifically to people with dyslexia read with ease, known as OpenDyslexic. The user can then access the other dyslexic-friendly features of the app.

Firstly, the user can adjust text size, and even add letter spacing, which helps them toward reading the text better. The read-aloud feature, built using Google AI's Text to Speech Library, as its name states, reads the text out using a realistic computer generated voice, which the user can follow along with. Last but certainly not least, the user can leverage the AI Summarizer which automatically gives a simple summary of intricate texts so it is easier and quicker to read, and more comprehensible overall. This part was built using Hugging Face's AI BART CNN Large Language Model (Bidirectional & Auto Regressive Transformer combined with a Convolutional Neural Network).

Challenges I ran into

Artificial Intelligence is a growing field and is continuously expanding everyday. I had some slight initial trouble when integrating these AI features into the application, especially the Google Machine Learning Kit's Optical Character Recognition part, as the extracted text from the image also needed to be simultaneously converted to the OpenDyslexic text format. However, after countless attempts, I was able to get the OCR to work. I was also able to get the other AI features to work smoothly as well.

Accomplishments & Learnings

I am very proud of the fact that BeyondWords can quite directly address the challenges faced by individuals with dyslexia, and has the potential to help serve as a reading companion for millions of people globally.

By creating a solution to enhance their reading experience, I have contributed significantly to breaking down barriers for those with dyslexia.

The incorporation of multiple dyslexic-friendly features, from text adjustments to summarization, gave me a significant understanding of the challenges faced by individuals with dyslexia. By providing a wide range of tools, I have created a comprehensive solution to enhance their overall reading experience.

What's next for BeyondWords

In Version 2.0, I am planning to implement reading evaluations using Reinforcement Learning. As the user continues to use the application, the Reinforcement Learning Model will start adapting text formatting to increasing the readability for that user. It will also support listening to the user reading out the material so that it can track the accuracy of the user's reading. I would also like to provide insights for each of the user's reading sessions, so that the user can read out the texts to find their accuracy and mispronounced or skipped words, and also provide analytics to the user periodically to help them gain insight into their overall reading performance.

Apart from this, I would also like to integrate Generative AI Question-Answer Model that will respond to the questions that the user has regarding the text, to help increase their comprehension of the material.

As they say, "A picture is worth a Thousand Words". I would also like to incorporate Generative AI for creating images about the text using models like OpenAI's Dall.E and Stable Diffusion XL. This will also help the user understand the meaning of the text better by relating it to the image they see.

Built With

  • android-studio
  • artificial-intelligence
  • bidirectional-autoregressive-transformer
  • google-ml-kit
  • google-text-to-speech
  • hugging-face-api
  • image-processing
  • machine-learning
  • optical-character-recognition
  • react-native
  • text-summarization
Share this project:

Updates