Reading Coach for dyslexia patients

Inspiration

70-80% of people with poor reading skills suffer from Dyslexia. Whilst it may not be very physically painful or require you to take a lot of Pills, it hits people on a different level. Often people with Dyslexia are labeled just too stupid to read, or too lazy to learn it, that is why a lot of the affected are embarassed to search for help. Especially adults who suffer from Dyslexia tend to hide it, and thus are unable to fight named disability. That is why we tried to create a virtual helping tool, which enables patients to learn reading at their own pace, and without needing to consult other persons. Although the application is mostly targeted towards Dyslexia patients, it is suitable for everyone who has reading issues.

What it does

We designed a quest-like app on iOS platform providing some challenges for users with Dyslexia, which find difficulties in associating the written words with the meanings. We provide a stress-free platform for the users to practice their reading. Our quests have different levels, from the beginning levels which are familiarizing different syllables, to more advanced levels which deal with vocabulary in daily life, making a gradually progress in imitating daily conversation, famous quotes from movies to pop songs. When users make continuous progress in the quests, it helps them to stimulate their ability to associate the written words with the meanings , strengthen their reading ability, and build up confidence in interpersonal communication.

How I built it

We started our ideas originally by designing the app with an IOS frontend and a backend develop by .Net Core hosted by Azure. First, when entering the app, the app will ask the user to sign up first so that the backend can either retrieve their user profile or create a new profile. After the backend receive their profile ID, it will generate quests for users according to their levels, which include the words/ sentences and the sound associate with them. The user will try to read a word/sentence out loud and press the button to record their speaking.Then, their recorded sound will be posted to the backend to compare the word and the sound user actually recorded (The comparison is done by calling Azure cognitive service speech to text), then they will be scored. Afterwards, the Users can play the sound in the app (The sound is retrieved by calling Azure cognitive service text to speech) and thus hear the correct pronounciation. There will be highlighted parts of the words/ sentences if there is some parts need the users to pay attention to. Then the user can repeat the test to see if they make any improvement.

Challenges I ran into

Right in the beginning we were confronted with the challenge of how to implement and design the communication between the user and the program. Obviously it cannot be done via written text, so the more or less only alternative was to use Text to Speech. As Dyslexic people often have problems with writing as well, we had to find a way for them to put in their solutions differently. We settled on using Speech recognition, which turned out be quite a challenge as well. Normally the user speaks in a very clean way when trying to communicate with a device, which Dislexya patients are capable of, but when reading something out load, they are often very slow and mispronounce a lot of words. That is why the speech recognition had to be able to locate these mistakes, while still recognizing the correct words. Furthermore, we had to find a special method to make the patients remember what they learned, as the conventional way already failed to do so. There is a multitude of approaches to this issue, but the most common one is the use of Phonetics and Symbolism. All in all, it was quite challenging to step into the shoes of a dyslexia patient, as reading is something so natural to the people who do not have problems with it.

Accomplishments that I'm proud of

Creating a working basis for a program that is designed to help people, and including methods which are adapted to methods that proved to be a useful help to people with dyslexia. Also the fact that we created this project as a team and all contributed. The work with Cognitive services also appeared like a big challenge to us at first, but we managed to include it in our program in a very useful and efficient way, which is something that we are really happy about.

I can get familiar with Microsoft Azure within an evening. Previously I was using Windows Servers technology, in this challenges I learn the skills to develop on Azure platform, and successfully develop some API connecting with Azure SQL database.

What I learned

When it comes to the non- technichal aspectsThrough the challenge, we got to know many types of disabilities, though some of them may not be well-known, we really need to pay more attention to them. We have done some research on Dyslexia, to know more about the symptoms, the cause of it, the challenges the patients are facing, and the possible ways to assist them. Time management and stress management within a limited time span, is something that we also had to learn and which was fairly difficult at first but with the time became more easy. For the technichal aspects, we learned about Rapid prototyping, work as a team when it comes to writing code and to separate the tasks between frontend and backend. Development and deployment with the Microsoft Azure platform. Also, we got to know the Azure cognitive service and learned how to make use of their services.

What's next for Reading Coach for dyslexia patients

Possibly enable the Application for different Languages and having a broader dictionary. Longer Sentences and a more diverse number of tasks. A further gamification to make the learning part more fun for the users. Integration with conversation agents. Developing a Web-Application and Android Application. A more diverse backend, possibly with a learning aspect about the users individual strengths and weaknesses.