Around the planet, the loss of social interaction has crept its way into the human population as mental-health has come to the forefront of our attention. In the midst of a pandemic that social distances us, we need solutions that improve our mental-health, now, more than ever.
What it does
Kare N Care leverages video streaming to connect people with each other in order to uplift our users' mental-health as well as providing professional emotional support. The app also is intended to analyze the emotions of the participant via voice recognition.
How we built it
We use React to build the user interface and host the application on Google App Engine. For emotions classification, we built a Neural Network model on Google Colab utilizing a compute instance hosted on Google Compute Engine. The API call is written in Python and hosted on Google Cloud Function. These will post the output using Dataflow with windows of 10s so the participant's voice can be analyzed in real time. Since working from home is a new trend, we use Twilio to help notify the supporters if they happen to be away from desks.
The data is collected from Spoken Emotion Recognition Datasets with 5 emotional levels ranging from angry to amused. The audio data is transformed to spectrogram images, then those images are used for training and validating the model. We use Convolutional Neural Network techniques to build the model.
Challenges we ran into
Accomplishments that we're proud of
We are proud of the front-end's aesthetic look as well as its functionality. We were able to utilize a lot of Google Cloud Services for the application and it made the deployment of the app much easier. We are also proud of the deep learning model that we have created from scratch.
What we learned
This Hackathon has helped us to spend time learning new technologies such as Twilio, Google Cloud Platforms, and try out different Machine Learning techniques for audio classification.
What's next for Kare N Care
We definitely will try to improve the accuracy rate by modifying the model. In the future, we also plan to improve the emotion recognition by introducing facial emotion using Computer Vision as well as adding more emotion classifications.