We took inspiration from the different categories on Spotify which separates music based on mood. We felt that it might be difficult for a person to exactly pinpoint his or her emotion, so we decided to create an AI to read their facial expressions.
What it does
The website takes a picture of the user's face and passes it into a trained convolutional neural network. Based on the output of the neural network, the Spotify API plays music from the user's account.
How we built it
We built the website using Bootstrap CSS and JS as well as jQuery. We built the Spotify API using the Spotify Java SDK and Spark for Java so that we can have a RESTful server. We built the neural network using Keras and TensorFlow in Python, and we did face detection with OpenCV.
Challenges we ran into
Accomplishments that we're proud of
We are proud that we were able to code the entire neural network from scratch and train it ourselves. We are also proud that we were able to make a clean web interface for a very powerful backend. Finally, we are proud of being able to complete the project in the time given.
What we learned
What's next for Music4U
Following this hackathon, we plan to deploy our app on a server so that it is accessible to everyone. We will also add functionality to the app so that users can give feedback about the performance of the AI and thus improve the accuracy of the AI. We also plan to update the UI and to add more features to the UI such as uploading a pre-saved image and a feedback system as already mentioned. We also have a 90% working iOS App prototype that we plan to publish on the app store.