There are millions of patients with dyslexia in the United States and across the world. Dyslexia is a particularly challenging learning disability for patients in elementary school - struggles with reading, math and learning sequences makes it very challenging to obtain a basic scholastic foundation.
As individuals with a profound interest in medicine and the healthcare technology space, we hoped to offer a solution to a significant yet unaddressed issue. After having met a patient with dyslexia recently, one of our members realized how significant of an impact it had on students. While this patient was a very bright individual, he struggled in school due to his reading level, challenges with learning sequences and understanding the order of mathematical symbols, and having troubles adjusting due to his difficulties with reading large paragraphs/books. We wanted to offer a solution which distills the task of reading to its basic components and which removes distractors and visual stresses to make reading a seamless and truly educational process.
What it does
The user uploads an image of the text they would like to read and using Python, pytesseract and Tesseract-OCR, this image is processed and converted into text which is then fed to the WebApp. The WebApp displays the text in easily managed chunks and highlights the current word (almost like a cursor) to facilitate learning. The user can read along as the cursor moves, and words which are challenging can be added to their personal list to be referred to at a later time. Furthermore, the user can view definitions and pronunciations to make their learning more meaningful. The number of difficult words and a Flesch-Kincaid Grade Level score are provided to help users get a gauge on how challenging the particular text they are reading is. Words which are difficult will also be identified to aid in the user's comprehension.
How we built it
Challenges we ran into
We wanted to make sure that the website we designed was accessible to dyslexics, so we consulted the British Dyslexia Association's Style Guide to ensure that fonts, spacings, colors and overall design were suitable for patients with dyslexia. This was a time-consuming but ultimately worthwhile process, as it improved the look and feel of our WebApp. It was also initially challenging to ferry data between the Python backend and the WebApp frontend, particularly with the transfer of images. This was resolved using HTTP and flask.
Accomplishments that we're proud of
We are proud that the WebApp can transcribe almost all of the images we submitted with few errors, and we hope that our users will gain something from our work.
What we learned
We learned how to apply speech-to-text, text-to-speech and image-to-text capabilities to a serious yet unaddressed problem.
What's next for Clarify
We hope to add additional functionality to break down math equations into more meaningful pieces, such as numbers, operators, and exponents. We would also like to offer our service to patients with dyslexia to evaluate how we can improve the user experience and fine-tune any issues with the OCR.