Dorothy - Smart Therapist, Caring Friend in Your Pocket
Mental health should not be an issue that is ignored. Therapy should be something that you don't loathe, or avoid. We know that mental health treatment is an important part of a child’s healing and that addressing the impact of trauma on the child significantly reduces harm and decreases the risk for future abuse. Therapy can help a child work through difficult, confusing and painful feelings in a safe setting. Therapy also provides children with the tools for going forward and leading healthy, productive lives. This is why we developed Dorothy.
What it does
Dorothy is AI-powered friend with intuitive understanding of young people's well-being, especially the youth. Dorothy is able to detect real-time emotion from tone of voice, facial recognition and sentimental analysis of text. Since Dorothy is geared towards adolescent, we make the app to be more interactive and engaging by providing features from music suggestion to self-guided meditation. We also add feature to locate the closest and most needed help resource based on the sentiment analysis and NLP
For young working adult, Dorothy uses NLP to be that hearing ears in a lonely night in New York City.
How we built it (with brain)
Next, we brainstorm what is the best approach to a quite challenging topic. Understanding that most adolescent feels more comfortable express themselves through spoken words than text, we decided to first work on the speech-to-text feature. To do so, we use Microsoft Azure Speech Recognition Api and flask to properly display the output.
To do so, we use TextBlob library in python. Dorothy is able to differentiate emotion based polarity and subjectivity.
Emotion Facial Recognition
We use OpenCv to have a real-time emotion detection. For Facial recognition, we use Tensorflow Emotion Recognition Face API. Dorothy is then able to enhance her emotion recognition features to encompass more emotion, such as Fear, Disgusting, Angry, Neutral, Happy, Surprise.
Natural Language Processing Help-Bot
We train a bot on Cornell movie dataset to be able to understand the english semantic as well as answer any question in the most natural way. Dorothy then incorporates Text Sentimental-Analysis and Emotion Facial Recognition to help user either talk their feeling out or guide them to the next proper step.
Brain and Psychology Domain Knowledge
Dorothy is built on the foundation of Cognitive Behavioral Therapy, Psychotherapist and Human Development understanding!
Challenges we ran into
Well. There were a lot. First one is probably getting lost in John Hopkins University.
Second is to find optimal solution that incorporates Cognitive Behavioral Therapy as well as Artificial Intelligence to solve a challenging problem.
Regarding technical challenges, there are some aspects we are less proficient than other but we were able to put every piece together efficiently!
Accomplishments that we're proud of
Probably most of the project. This is an idea that we have for a long time and finally we bring into fruition. However, we are most proud of the fact that we help solve a problem that we are both concern about!
What we learned
What's next for Dorothy - Your Digital Friend
WOW! we have a lot more in store for Dorothy.
- Have a MVP for iOS and Android app
- I (Hang) am personally super excited to add in Tone Emotion Detection. That is why we initially come up with the idea to do speech-to-text feature. Tone of voice shows a wider range of emotion. Then, we will enhance Dorothy's ability to understand emotion further.
- We would love to take the NLP further to be more suitable for the audience. Using a bigger and more appropriate dataset will help Dorothy to sound professional and caring like a true friend.
- Finally, we want to create a wellness report for the user based on the conversation. For example, if there is a trend in their mood getting better and what is the reason for such. Ultimately, I am really interested in the collecting a dataset that helps researcher and scientist understand the early sign of depression in adolescent!