Dance is a beautiful art form and many people want to learn it, as shown by the growing number of dance studios. With our new physically-distanced world, I can no longer take advantage of that. Learning dance moves from choreography videos is a wonderful tool, but videos cannot tell you if you are doing a dance move correctly. I want people to experience the benefits of a real-life coach with the convenience of on-demand video.
DANCE uses artificial intelligence to detect people’s dance moves. It evaluates and tracks how a person is dancing compared to how the dance should be performed, so users of DANCE learn how to dance better and assess their progress in learning dances.
Challenge 1 - I faced major hurdles when attempting to implement the PoseNet model, especially due to the complexity of extracting frames of a webcam/video and applying it. Some relatively simple features, like plotting the points of where PoseNet estimated limbs to be, actually required a lot of tinkering to erase the points after a certain period (so that the screen doesn’t become cluttered).
Challenge 2 - I have encountered various issues that may seem trivial at first glance, but turned out to have consumed large amounts of time and energy. For example, I spent hours attempting to fix a problem caused by a misplaced iteration variable in a for-loop as I initially thought the problem was caused by something else. I also spent a long time figuring out async/await statements for a few functions to solve a problem that could be solved with a simple if-else statement. Through rigorous checking and testing I finally understood my mistake also a google meet with Jonathan Lei helped me to sort out most parts of it.
Challenge 3- Another issue I faced was the time issue. Though timings was known to me but I am exhausted while working on this as the timings were just opposite. Being in India it was really a challenge as well as pride for me to participate and finally make an entry
Accomplishments that I am proud of
I am exceptionally proud of how my project can better teach people dance and enhance learning from choreography videos/tutorials. At first one mentor helped me to develop a good outlook visuals of my website. Then I took it even farther and added beautiful micro-interactions. Subtle CSS transitions were added to most elements. If there was a user interface award in this hackathon, I am confident that I am a strong contenders. In addition, applying the PoseNet model on both a video and live webcam footage required creativity and robust programming. Comparing the two with an algorithm was also an exceptionally difficult challenge that I conquered!
What I learned
Working with PoseNet introduced all of me to a complex AI algorithm that piqued my interest and served as a great introduction to working with big datasets. I also took away different things from this project depending on their focuses. For example, I learned how to create CSS animations and use JQuery to show/hide different elements. working on the backend improved my skills with Firebase and handled difficult interactions with PoseNet objects.
What's next for DANCE?
From the data I glean from our user’s dance moves and perhaps a few more hours, I would be able to develop custom dance plans with specific instruction with which users can improve their weak points. With the same data, perhaps I can write something that choreographs new dances using the user’s strengths and moves they have already learned.