Many issues plague our society amidst the COVID-19 pandemic. Some scream louder than others, but mental health remains a silent, yet dangerous, consequence of lack of human interaction. We at Mooder explore the possibilities of utilizing AI to aid in the battle against mental health. By pairing emotion classification with clinical questionnaires, we hope to give more context to the patient's reponses. Not only are we exploring ways to improve the current system, but we were inspired to build something that would allow clinicians access to more data for longitudinal studies. With more data, one could look at the trends of patients' mental health to allocate more resources during tough seasons, or just better understand mental health.

What it does

Mooder is a web application that pairs a traditional depression screening tool (PHQ-9) with an emotion classifier through the patient's webcam. As the patient fills out the questionnaire, their emotion throughout each question is recorded. This gives more context to the questionnaire which otherwise would have been lost. In addition to facial emotion classification, we also detect the emotion of text during three short prompts as well as extract keywords. This increases the efficiency and effectiveness of the screening. We then provide the clinician with a detailed report of the answers and the patient's respective emotions during each question.

How we built it

The facial emotion classifier was trained on the FER-2013 dataset which had 7 emotion labels. We utilized a model from ( but found that retraining the network for our specific case was more effective as we only required 3 of the 7 labels. For the text-based emotion classification and keyword extraction, we utilized an API (ParallelDots). We utilized Flask, HTML, and CSS to build a demo web-app locally.

Challenges we ran into

HTML and CSS was difficult. Neither of us has any experience with HTML and CSS. We barely had experience with Flask. The neural network and python-based image processing of images was our strong suit, but creating a web app locally was a steep learning curve. We also are in slightly different time zones, so coordinating the project was difficult.

Accomplishments that we're proud of

Mental health is a big issue especially during times of loneliness - and we are proud to work on a project that tackles such an important topic. We are also proud of all that we have learned throughout the process. We are excited that we were able to create our first local web app!!

What we learned

Outside of the technical skills that we had to pick up to complete this project, from PowerPoint creation to learning new languages, we learned how important time management was. It's important to plan out a project well and set realistic goals. It's also important to know what tools you have available!

What's next for Mooder

We hope to continue the fight for mental health. As technology continues to improve, we expect many applications like Mooder to greatly benefit the healthcare system. Ideally, Mooder would aid counselors and other healthcare professionals with screening their patients to increase the efficiency and effectiveness of their treatment.

Built With

Share this project: