Inspiration

During lock down, mental health has become more and more important. Our moods are easily affected by our daily activities. While there is only so much we can do with a project built in two days, we realized that there was a way for us to have an impact, and that is through music.

What it does

MoodZik is a program that will use a small sample song that is only a few seconds long, and compose a song from scratch using Machine Learning, after using that same technology to extract the users emotion from their face through their webcam. The music will go along with their mood, and they have to option to re scan their face when they hopefully feel better, which would be success in our estimation.

How we built it

The first part: We train a model on detecting facial emotions, using a data-set called fer provided by Kaggle, using CNN (Convolutional Neural Networks), and we store the trained model. The second part: We take the webcam input, detect the face in it and output a square surrounding it, then use our model to identify the emotion expressed. We then output the emotion next to the face's square. The third part: We made a program that gets inspired by a small sized sample song, to then utilizes machine learning to compose a new song from scratch. Then, we made another program that uses PyGame to play the song, and provide feautures such as pausing and unpausing. The fourth part: We made a web app using flask, html and css, that contains a page for recording an emotion, and a page where you can play, pause, or replay the song generated. There is also a button for regenerating another song by re scanning the webcam input for the user's emotions.

Challenges we ran into

Creating and training our model was difficult and required a lot of debugging and testing. The hardest part was finding and managing the data-set csv files. It was also quite challenging to generate the music file from scratch, especially since we had to use a multitude of libraries, which caused a lot of compatibility issues. The most difficult part was getting all of those parts to work with each other, by sending files for other scripts to use, especially since each of us worked on different parts of the project.

What I learned

We learned a lot about developing a web app, and working with machine learning. But mostly, we learned a lot about working as a team, especially using git.

What's next for MoodZik

We still have a long way to go, especially in terms of optimization, as our project could be made to be a lot faster. We also want to add more features to our web app, and make an even more compelling GUI.

Share this project:

Updates