HAPI

Inspiration

The birth of a child is a powerful emotional experience, bringing feelings of overwhelming joy and excitement, but in some cases also tearfulness, hopelessness, and feelings of suicide. Maternal depression is a serious issue with a prevalence rate of 50%, with 85% of the cases not being diagnosed.

In the wake of International Women’s Day, we decided to empower pregnant women and new mothers. Pregnancy is an important milestone that most women will experience in their lifetime, and we would like to minimise mental health issues that women may face during the process. Our vision is to improve diagnosis of depression and anxiety amongst expecting and new mothers.

What it does

Hapi acts as a complete pregnancy platform, with a focus on both mother and child.

Our primary focus involves monitoring vital indicators such as sleep, activity, heart rate and emotion through text and image analysis. Following user consent, image analysis of the mother’s expressions using our API allows detection of certain emotions which may indicate low mood and depression. Once a continuous low mood has been detected, the mothers will be notified with the gold-standard questionnaire used for post-partum depression. After this, the user can be appropriately directed towards their midwife or GP, if required.

As well as measuring the mentioned parameters, Hapi also provides feedback to the mother in response and generalised pregnancy advice and articles.

How we built it

The image classification and sentiment detection were both developed in python. The text sentiment analysis utilises the VaderSentiment library to carry out sentiment analysis on a diary entry to look for negative, neutral and positive terms and returns a compounded score which would be used by the backend to record the emotional state of the user.

The image analysis was initially developed using the Cohn-Kanade dataset for expressions. This was used to train a CNN with the fisherfaces of each image and was able to display the emotions detected live on a camera feed. We then however switched to the Microsoft Azure Faces API as we were struggling with accuracy due to the small size of our training dataset. This also allowed us to detect other key indicators from each image such as age, gender, headPose, smile and emotion amongst others. For the purpose of the demo this is then overlaid onto the taken image showing the detected face and the gender, age and emotion. The app however will take the relative scoring of each emotion to create a happy score used by the backend of the app.

Accomplishments that we're proud of

We are proud of Hapi. We have robust emotion analysis through two methods: image classification and sentiment detection which the backend of the app will be able to use to monitor the mental health of the user. We have also developed a sophisticated user interface demoing the full capabilities of the final app, created from scratch over the two days which incorporates some existing diagnosis practice which we discovered through extensive research.

What we learned

This weekend taught us methods in rapid app prototyping and the current criteria used for the diagnosis of maternal depression. We were also alerted on the staggering statistics surrounding prevalence and missed cases. We learnt a lot about the implementation of image and text analysis in python and the various methods such as neural networks and APIs.

What's next for HAPI

Our future vision is to expand and improve Hapi. We would like to increase the number of parameters we measure, implementing measures such as blood pressure and attention and memory deficits, shown to be accurate indicators of depression. Furthermore, we’d like to implement in-depth user testing to fine-tune the app for our user’s experience. This will involve training the algorithm so that it is able to detect more subtle changes in facial expression.

Built With

• adobe-illustrator
• adobe-xd
• python