Music is a very important part of everyone's day. It helps you relax and get going but sometimes it's just too difficult to find the right music. It's really hard to find the right playlist or album that goes with your mood and tends to hinder with our productivity in some cases. To tackle this problem and to make working environment more productive, we came up with EmoFy.

What it does

EmoFy is a web app that makes use of deep learning to provide users with the best music based on their mood without any hassle. We made use of different web technologies to build the application along with a custom deep learning model that is running in the background. The model helps in detecting human emotion in real time and using the output that we get i.e. the emotion/mood, we recommend the user songs that they can play using Spotify without even opening the application.

How we built it

  • We started off by building our model. Using a dataset from Kaggle we built a custom CNN architecture in order to detect the human emotions.
  • Once we had our model ready, we moved on to build our website.
  • We started off by writing the backend which is built using Django. We made use of Opencv to get the camera functionalities and to detect human emotions in real time. We loaded the model using tensorflow and keras which runs in the background and helps in emotion detection.
  • Then we moved on to build our front end. The process started off by creating a Figma prototype. Once we had our prototype ready, we started building the frontend.
  • We used Bootstrap to build our website based on the designs and created the necessary pages. We then went on to connect the backend and the front end and the application was ready.

Challenges we ran into

One of the biggest challenge that we faced was during the model building process. We wanted to come up with a custom architecture for our model instead of going for a model that is pre-built. So that took a lot of experimenting to come up with an architecture that was good at giving predictions. Other than this we ran into some minor opencv errors but we dealt with them and got the application working without any errors and also made the website responsive.

Accomplishments that we're proud of

  • We were able to come up with a custom model architecture for emotion detection rather than using any pre-trained model.
  • We successfully integrated our ML model with a web application. We were able to come up with a model that had 65% testing accuracy and 71% training accuracy which was great as the best model that was built was known to have 70% accuracy on training data and 69% accuracy on testing data. Overall the model performed really well given that we built a completely custom model.
  • We were able to integrate Spotify in our web application.
  • We were able to use Django backend instead of the normal Flask backend.
  • We also went with a more complete way of working on the project by using Github effectively, using the project boards and issues and also took the approach of designing and then building which we are really proud of.

What we learned

To come up with the custom model architecture we had to go through various research papers of pre existing models such as MobileNet, VGG16 and ResNet in order to understand how these worked. Most of us in the team were quite unfamiliar with working of Django so we had to go through the documentations in order to understand the working and build the model as we did not want to use flask for the web app this time. We also learnt integrating Opencv with our Django backend which was not a very hard task to do. We gave more importance to User interface and experience(UI/UX) this time and hence spent some time designing the web application and logo.

What's next for EmoFy

We understand that Spotify is not the only place where people go to stream music hence we plan to bring in more services such as Youtube music and apple music on our application. We are also working on building a reward system which will be basically token specific to our app deployed on Ethereum so that people can be rewarded for their contribution to the website, this contribution can be in form of images for the model or music or in any specific part of the website.

Built With

Share this project: