Autism Treatment Assistance

Inspiration

Autistic kids throughout the world suffer from symptoms that are very hard to treat and they are required to go though long and extensive therapists to help them require some of the skills, mostly social, they lack. A common autism symptom that psychologists are aiming to treat using different methods is lack of facial emotion expression in correlation with the content of speech. Autism Treatment Assistance is an attempt to offer therapists the power of data science and computers to hopefully tackle this problem.

What it does

Autism Treatment Assistance aims to give a complete and organized analysis of the facial expressions and how they change depending on the content of the speech, down to sentences. Therapists need to record their patients face during the therapy and later on using this program observe how and when the facial expressions change according to the content of the speech and evaluate the progress in the long term.

How I built it

First convert the video to audio and then use the Watson Speech-to-Text service to convert speeches to text. We then use the IBM tone analyzer to analyze the tone of the speech down to sentences. The emotions include joy, sadness, disgust, anger, and fear. We search and index the audio using the sentences extracted and with the correct timestamps we go on to use Microsoft Emotion API to detect emotions in the facial expressions of the patient during the sentences we analyzed before using tone of the speech. We then compare the result of both facial and tone expression and will give an analysis of which sentences missed the objective and marks the areas of strength.

Challenges I ran into

Dealing with big data and handling multiple parts of the code in parallel to get good results. Using and working with data science services and APIs seriously for the first time.

Accomplishments that I'm proud of

I'm proud that I've been able t build something that aims to solve a real world problem that are affecting people throughout of the world. Autism Treatment Assistance helps autistic kids improve in this kind of therapy and will be able to be more integrated in the society and thrive in their education.