Inspiration

Dance is a beautiful art form and many people want to learn it, as shown by the growing number of dance studios. With our new physically-distanced world, I can no longer take advantage of that. Learning dance moves from choreography videos is a wonderful tool, but videos cannot tell you if you are doing a dance move correctly. I want people to experience the benefits of a real-life coach with the convenience of on-demand video.

Working

DANCE uses artificial intelligence to detect people’s dance moves. It evaluates and tracks how a person is dancing compared to how the dance should be performed, so users of DANCE learn how to dance better and assess their progress in learning dances.

Making

Beginning with my vision for an AI dance tutorial, I designed a wireframe mockup of the flow of my app and the functionality of the pose comparison algorithm. On the front end, I used HTML & CSS to style the site and extensive Javascript to import and implement. I used the Posenet library to implement AI-based pose detection. For the backend, I worked on retrieving videos and a list of timestamp markers. Tutorial videos loop according to the timestamps and those markers can be skipped, ignored, and replayed. Feedback from a webcam is taken and used to display a live score. The score is calculated by comparing the data from the webcam to the data from the original/professional dance video.

Challenges

Challenge 1 - I faced major hurdles when attempting to implement the PoseNet model, especially due to the complexity of extracting frames of a webcam/video and applying it. Some relatively simple features, like plotting the points of where PoseNet estimated limbs to be, actually required a lot of tinkering to erase the points after a certain period (so that the screen doesn’t become cluttered).

Challenge 2 - I have encountered various issues that may seem trivial at first glance, but turned out to have consumed large amounts of time and energy. For example, I spent hours attempting to fix a problem caused by a misplaced iteration variable in a for-loop as I initially thought the problem was caused by something else. I also spent a long time figuring out async/await statements for a few functions to solve a problem that could be solved with a simple if-else statement. Through rigorous checking and testing I finally understood my mistake also a google meet with Jonathan Lei helped me to sort out most parts of it.

Challenge 3- Another issue I faced was the time issue. Though timings was known to me but I am exhausted while working on this as the timings were just opposite. Being in India it was really a challenge as well as pride for me to participate and finally make an entry

Accomplishments that I am proud of

I am exceptionally proud of how my project can better teach people dance and enhance learning from choreography videos/tutorials. At first one mentor helped me to develop a good outlook visuals of my website. Then I took it even farther and added beautiful micro-interactions. Subtle CSS transitions were added to most elements. If there was a user interface award in this hackathon, I am confident that I am a strong contenders. In addition, applying the PoseNet model on both a video and live webcam footage required creativity and robust programming. Comparing the two with an algorithm was also an exceptionally difficult challenge that I conquered!

What I learned

Working with PoseNet introduced all of me to a complex AI algorithm that piqued my interest and served as a great introduction to working with big datasets. I also took away different things from this project depending on their focuses. For example, I learned how to create CSS animations and use JQuery to show/hide different elements. working on the backend improved my skills with Firebase and handled difficult interactions with PoseNet objects.

What's next for DANCE?

From the data I glean from our user’s dance moves and perhaps a few more hours, I would be able to develop custom dance plans with specific instruction with which users can improve their weak points. With the same data, perhaps I can write something that choreographs new dances using the user’s strengths and moves they have already learned.

Built With

Share this project:

Updates