Pose | Devpost

Inspiration

Our inspiration for the project came from the fact that we don't know how to dance and we believe there are many individuals like us, who would love to learn to dance but don't have easy access to dance lessons. Furthermore, our application can also be used for other skills that require certain repetitive body movements. It can be used to learn different sports moves, martial art skills, and any other such skill.

What it does

What we do is that we process video files from sources such as YouTube or even ones that you could upload and then we generate a silhouette that mimics the movement of the figure in the video almost exactly. Then we used real time camera data and process that in real time that produces a silhouette in real time, and so you can try matching your silhouette as close as possible to the video silhouette. For example you could get a silhouette from say Kobe Bryant's basketball shooting form, and then practice your shooting form in real time and try to match it to Kobe's. When you don't match the silhouette's position there is a bored indicator that indicate that you didn't match.

How we built it

We used a deep learning model implemented in TensorFlow.js and integrated it with a web application that can take a video file and can also take in real time camera feed and apply the pose estimation model that creates the silhouettes.

Challenges we ran into

Challenges we ran into really derived from our inexperience in JavaScript which hurt us since we tried building a web application. We also ran into challenges with trying to process video data and alter it so that it made our application aesthetically better.

Accomplishments that we're proud of

We are also proud to have been able to build a project that was equally enjoyable as it was challenging. Furthermore, the fact that we were able to make something that addresses the issue that we were to solve was pretty enriching.

What we learned

We learned web technologies that we were not well versed in before such as JavaScript and css. We also learned that it was possible to integrate a TensorFlow model into a JavaScript application.

What's next for Pose

The next step for Pose is to develop a robust scoring mechanism that depends on the pose silhouettes generated by the model. The scoring metric would be gauged by the separation between the two silhouettes.