Inspiration
During the pandemic lockdown, many people were confined to their homes and unable to hang out with their friends. This project was inspired by online dance parties where people where people would join a zoom call and dance together. Our projects takes this to another level by using Machine Learning to allow people to join a party and dance, but as stickmen!
What it does
Our app has two functionalities. The main function is that the app allows users to take their camera feed, and through machine learning we are able to re-create the movements of the dancer in a stickman that is drawn on to the webpage. This helps ensure that anyone can join the dance and have fun, friend or strangers alike! The second function is that it allows the user to connect their device to the server using spotify and stream music from there! The app fully integrates the controls that are needed to control the grooves, now all you have to do is show us the moves!
How we built it
We built this project starting with an React template, and we slowly imported several functionalities (Spotify SDK, a google cloud server which ran the ML data, and socket.io). However, the javascript functions ran too slowly so we ported our project over to HTML and ran it using a node server instead. From there we used CSS to clean up the UI and now it is a project we are very proud of!
Challenges we ran into
When we built this project using React, it worked well at first, but it fell out of favor due to being unable to run complex machine learning functions fast enough. We had to port our entire project over to HTML. We also had a fair bit of trouble tweaking the Spotify so it worked every time and making sure it did not log too many errors in the console. We also needed a new Spotify key every time so we also had to implement Spotify authentication. We had to learn the two.js documentation in order to get the stick figures animated. The canvas also had a fair share of problems with positioning.
Accomplishments that we're proud of
We are very proud that we were able to build this application from the ground up into a useable webapp. The UI was very clean and we are very proud we were able to work together very well as front-end and back-end developers. We had no idea how to build this project from the start, but we learned how to connect the ML and animate the stick figures using various javascript modules. This was also our first machine learning project!
What we learned
We had no idea how to build this project at the start but we browsed through lots of documentation and learned a lot of new knowledge about programming in javascript. We also solidified our understanding of javascript by writing so much of it. We learned a lot about how to implement back-end and APIs into our project.
What's next for Digital Dance Party
We hope to be able to improve the product after the hackathon by learning more two.js to better animate the stick figures. We also hope to improve the user interface as well as add more functionality such as private rooms to our project.
Log in or sign up for Devpost to join the conversation.