In today's fast-paced technological society, issues of mental health have often been ignored. Mental health issues prevent employees from achieving their full potential both in their personal lives and in the workplace. In fact, the estimated cost of mental health issues to the Canadian economy is estimated to exceed $50 billion annually. Employers specifically suffer over $6 billion in lost productivity from absenteeism, presenteeism, and turnover. With most adults spending more of their waking hours at work than anywhere else, addressing issues of mental health at work is vitally important for all people in Canada.

What it does

Our product, SentimentalBoi, leverages machine learning advances to empower people to take hold of their own mental health through second-by-second mood tracking and insightful data analysis dashboards. By showing users a breakdown of their mood by time and day, we enable them to know exactly when and why they aren’t feeling at their best!

The mood tracking is achieved through the user’s webcam. The user’s face is constantly tracked using a machine learning sentiment analysis approach. When the user is feeling happy, sad, angry, or scared, SentimentalBoi updates the user’s personalized dashboard to show both the time and frequency of these emotions!

For employers that are invested in the mental health of their teams, SentimentalBoi provides an intuitive and centralized dashboard to obtain the insight necessary to make proactive workplace changes.

How I built it

We built this application with the use of open-cv and Azure Cognitive Services (ACS) and ElasticSearch. Our application used computer vision on the images collected from our webcam and performs an API call to the ACS. The output is then fed as a json object with a timestamp into an elasticsearch which indexes the data for us. We made use of Kibana to generate dashboards and visualizations which consumes the data from the ElasticSearch Queries.

Challenges I ran into

We initially wanted to build the whole application on AWS with AWS ElasticSearch, Lambda, Computer Vision and also host the website on AWS. However, we soon realized that AWS Educate tier does not support ElasticSearch which threw a wrench in our project. We resorted to using Elastic Cloud hosting along with a localhost Webcam application. Most of the challenge we encountered were related to using different platforms.

Accomplishments that I'm proud of

We’re proud that we were able to quickly leverage the services provided by Azure to provide us with the sentiment analysis we needed. We are also proud of being able to visualise sentiment data in a way that provides new insight into the user’s day to day life.

What I learned

We learned a lot about computer vision since for many of us, it was our exposure to that specific technology. Many of us also did not have any experience working with ElasticSearch or Kibana. Hosting our ElasticSearch as a Cloud Backend was really key to our project so learning about interacting with cloud hosted applications was really cool.

What's next for SentimentalBoi

The next step for SentimentalBoi would be to deploy it in order to make it fully accessible to as many people as possible. We also believe that this kind of tech could have many other applications in analysing the emotional responses of large crowds, which would be the next area where we would expand. Some such analysis could then be used to monitor the success of ad campaigns, movie trailers in the movie theaters or even to dynamically customize music being played at a concert based on emotions of the crowd.

Built With

Share this project: