We have seen veteran families where a person is too tired to be angry, he talks about the innocence and honor of young soldiers. Then he may suddenly cry and the heart of the family members breaks all over again. We developed this application where a family member can click a picture of the person and using the Bose soundtouch, it will play music based their emotional state without bothering them.

We have a web application where we click the image of the veteran and recognise the current emotion on the face of the person. After recognising it, we select a playlist based on their emotion without bothering them and help them calm down. This application has been developed keeping in mind to control the anxiety issues of the veterans and help them make a smart home.

What it does

It takes a picture of a veteran and any other person too from a live running video in our application and recognize the mood of the person on the basis of our trained model and recommends the person song playlist through Spotify, which can improve the mental state and anxiety of the person.

How we built it

Our frontend is built using Javascript and backend is built using Python. We have a constant recording video in our application and user can take a picture whenever they want to. The picture goes through ibm watson where the face portion is received then we test the image against our trained model for the emotion recognition. Our application then identifies the emotion of the person we recommend music from Spotify by identifying counter emotional songs to make them feel good and play it on Bose Soundtouch.

Challenges we ran into

Bose integration with Spotify and synchronization of the event with the backend, bose and spotify.

Accomplishments that we're proud of

We built a home automation project helping veterans and they can listen to songs according to their mood and cheer themselves up.

What we learned

We learnt pretty new things during this hackathon. We learnt about Spotify and bose soundtouch integration. we also learnt about how we can use machine learning to recommend songs on basis of person's current emotions.

What's next for moodSwinger

We can improve the performance by increasing the training data set for more accurate results and we can also use speech recognition along with emotion recognition to have a better perspective of behaviour.

Share this project: