Just Dance Dance Revolution

Landmark-Analysis of YouTube Video
Fortnite!!!

--Third Place Finish in the Arts & Games Category--

Program isn't lagging in above video - Dancer just lacks dancing ability

Inspiration

We were inspired by the joy of learning new dances but recognized the pitfalls of YouTube tutorials which result in a tedious process of pausing, rewinding, and confusion. We aimed to create a way for people at home to freely, easily, and effectively learn dances of choice. We envisioned a software solution capable of seamlessly overlaying YouTube dances while providing real-time analysis of their dancing progress. By integrating computer vision, our software would track the user's movements, offering feedback on technique, timing, and posture.

What it does

Our software takes any YouTube dance video link, conducts a deep analysis of the dancer's body coordinates, and organizes it into a large data frame. It then uses computer vision to provide a real-time position analysis of a user's dance positioning while displaying a landmark-based overlay of the YouTube dancer on the screen. After this process, the program computes summary statistics of the user's dance compared to the YouTube dance. A linear regression analysis is then applied to give a score to the user on their accuracy in the dance.

How we built it

We applied computer vision techniques to process live 3D landmark coordinates. We then stored the vast amounts of data into pandas data frames. We then designed a method of comparing the user's movements to those of the YouTube dancer with angle computations, landmark positioning, and linear regression. Additionally, we had a frame-by-frame analysis of the input YouTube video and used computer vision techniques to extract coordinate data for creating an overlay. This data was then compiled into a data frame which we used for overlay and comparison.

Challenges we ran into

The main difficulty was the coordination of many different types of data formats with various types of library modules. We also had difficulties resolving merge conflicts. However, we were able to quickly able to resolve issues with appropriate team communication and study of API documentation.

Accomplishments that we're proud of

Scaling the dance overlay onto live users using an AI model
Management of large amounts of complex data in between multiple files
Creating a successful application that could analyze how well a user dances ## What we learned We learned about proper product timeline management and smoother use of version control. ## What's next for Just Dance Dance Revolution We plan to increase the accuracy of dance analysis, as well as develop a more robust UI.