Sonia and I have been best friends since high school but now live on opposite sides of the country to attend college. We missed spending time with each other and wanted to emulate an in-person experience. We also both have been out of breath walking around our respective campuses and considered the benefits regular physical activity (ie. dancing) provides. We were inspired by Just Dance and Zoom to create a platform that combines the two to connect users to each other, different cultures, and to the world of dance.

What it does

It is a video conferencing website where an individual can join a room with a friend and follow a dance video with them while their body is tracked and compared to the dance video to result in a score. Similar to Just Dance, consumers can compete with their friends, but Jam allows their friends to be anywhere in the world. Although currently, we haven't been able to integrate the machine learning aspect with our website, we were able to do a proof of concept within the 24-hour time constraint.

How we built it

We split the project up, frontend and backend, Sonia was working on the front end in which she used HTML, JavaScript, and will to make a UI/UX user-friendly video conference platform that is fun and easy to use. While Pragya and I were on the backend, where we used python, OpenPose, and google colab with the intent of using OpenPose to place key points on users' bodies that would be tracked based on the movement of the key points, scored, and averaged out to make up the final scores after each competition.

Challenges we ran into

We ran into many challenges but a huge hurdle we had to overcome was downloading the necessary libraries and packages needed to run and integrate the platform. Issues arose with downloading and implementing Node, Cmake, and OpenPose. We were able to overcome these issues with the help of online forums and Technica mentors. Another challenge we ran into was the fact that Ameenat and Pragya were attending the in-person event while Sonia was virtual in California with a 3-hour time difference. We FaceTimed and texted a lot throughout the event to make sure we were on the same page.

Accomplishments that we're proud of

We are proud of the fact that we were able to pick up something new and learn so much from it. It was very challenging to attempt to learn about most of what we used to help build the project in less than 24 hours because we had little to no experience but after some guidance and teamwork we were able to persevere!

What we learned

We learned a lot about machine learning and how models are trained and applied to various situations. This is the first time we have used machine learning with images and videos so it was very interesting to see how that data was different than data points we have worked with in the past. We also learned about Twilio and how it can be integrated into a website.

What's next for Jam

We would like to create join codes for rooms in Jam and have live feedback for scoring the users performance.

Built With

Share this project: