Inspiration
Have you ever listened to music and felt your heart pound? Well that's because music affects your heart rate via a cool phenomenon called sympatho-respiratory coupling.
What it does
This application takes advantage of sympatho-respiratory coupling by steering the user's heart rate using music to achieve a desired heart rate.
How we built it
Training data on user heart rate and music information such as tempo and pitch was fed into a pytorch neural network to develop a model to predict the user's heart rate. This model was then used to predict what the rate of change of heart rate would be for a given set of songs, music is then selected from this list by selecting whatever song achieves the greatest predicted change in heart rate in the direction of the target heart rate.
Challenges we ran into
1.) Getting heart rate data: We tried a variety of approaches, ranging from a Garmin vivofit3, Apple Watch, and Fitbit Luxe, only the fitbit had the ability to get data without developing a native application. However, fitbit data is peculiar in that the information is not sent from the device, but is rather sent to Fitbit's website, where it must then be accessed by an application registered to make API calls to Fitbit's website. This indirect approach required learning a great deal about authentication.
2.)Training a Neural Network: We did not have a pre-trained neural network available, we had to train our own. Fortunately pytorch comes with templates for neural networks that just require training data. As a result our team members had to manually input their heart rates while listening to songs to ensure fidelity in the data.
3.)Bringing the pieces together: Because heart rate data must be retrieved using a web application, and our neural network ran using pytorch, we had to use Flask to run bring our python scripts into Javascript so that we could run the neural network.
Accomplishments that we're proud of:
It works, it really truly works. It's a magical feeling when our algorithm picks a song for you that just seems to get your heart pumping if you want to feel excited and soothing when you want to feel calm.
What we learned:
We learned there is a strong correlation between tempo and the change of heart rate. Songs that have faster beats tend to produce less change in heart rate. This is likely because the heartbeat cannot synchronize with music that is too out of sync with itself.
What's next for HarmonAIze:
Our current neural network is fairly primitive and more closely resembles linear regression. Upon inspecting correlations in our data clusters in different regions even for the same tempo. We suspect this might have to do with time signature.
Log in or sign up for Devpost to join the conversation.