EmoSense

Inspiration

A few months ago, I watched the film "Lie to Me". The plot of the film is built on emotion detection, and it inspired me to create an AI to detect emotions in photos.

What it does

Emotions are an essential part of our mental and physical health, so we created the app to process images and detect emotions on every face in the image. The app supports six types of emotions: anger, disgust, fear, happiness, sadness, and neutral emotion. There are two options for how you can upload an image. The first is to upload it from a local file from your computer, and the second option is to provide a link to this image.

How we built it

We used various technologies for this project. First, we used Python and TensorFlow to train a neural network. We used Kaggle servers(link to notebook: TODO). I found a dataset called CREMA-D on Kaggle. There are lots of videos with actors saying small sentences with different emotions. Due to the fact that we needed pictures and not videos, we extracted most of the frames from videos, separated faces, resized part of a frame with the face to 48x48px, and grayscaled it. I uploaded this dataset on Kaggle and made it public, so if anyone needs it, here is a link(link: TODO). After model training, we made an app using Electron and Javascript. However, we couldn't combine Electron, openCV, and TensorFlow, so our app calls a Python script to do all the work with photos.

Challenges we ran into

There were a few big challenges. For instance, we couldn't use openCV in our js app. We lost a tremendous amount of time on this problem, but we still had to take a different approach, and we still have no idea why we are receiving this error. Also, a challenge was to decide which model structure to use.

Accomplishments that we're proud of

We are proud that we managed to complete the project. And that we completed it on time)

What we learned

We learn a lot about TensorFlow and artificial neural networks. Also, we gained experience creating desktop applications using Electron and Javascript.