When were things created and who are we?

Everything created in the project was done this weekend (2/20 - 21)! We are Sarang Goel (a 9th grader) and Zoya Hussain (8th grader) and we both are beginners at coding with 1-2 years of experience!


Our inspiration for this project arose from some internet surfing and research. During quarantine, many, if not most people partook in exercises and yoga during lockdowns. And so, we created a tool to assist others in form and manner.

What it does

Our ML model utilizes computer vision to accurately determine fitness performance, analyze form and posture in a variety of exercises, and collect user data for classification purposes. All of this can be implemented into our website for simplicity and easy use.

How we built it

ML Model: In order to determine how well a person is doing a certain exercise, we had to use a pose estimation machine learning model. Due to time constraints, we used a pre-trained pose estimation machine learning model for our project called MPII. The MPII model was trained on over 25,000 images containing over 40,000 people with annotated body joints. Overall, the dataset covers 410 human activities, making it useful for human pose estimation, especially for exercise-related activities. However, during testing, the model did not perform well for videos where someone is performing a horizontal exercise (such as a plank) so the model was optimized for these types of exercises for better accuracies. When the model is applied to the video, the python code we created breaks the video up into separate frames, and the model is run on each frame. The model estimates various key points of the person on each frame, such as the neck, right shoulder, left shoulder, and so on for a total of 15 keypoints counted from 0 to 14. The key points are connected to create a skeleton of the body. The model can accurately identify these skeletons on various exercises that we performed the model on, such as jumping jacks, lunges, pushups in rotation, and tricep dips with a chair. Furthermore, the model was used to give real-time input to the person exercising about how well they are doing the specific exercise. The model was able to determine feedback on the user's performance by using the skeleton. We first extensively researched the common ways people do certain exercises correctly and wrongly and wrote corresponding python code to use the slope of specific skeletal features and angles between different skeletal features to allow the model to make accurate predictions. All of this can be done on any computer but requires an NVIDIA GPU for real-time purposes. We found that our model was highly accurate.

Website Model: The website model was prototyped and coded in HTML. It utilizes 4 different screens: Dashboard, Preferences, Your Progress, and Competition Mode. Additionally, it has multiple preloaded exercise routines that can be readily paired with the model for easy implementation. The website is coded to readily display feedback to the user for real-time use and correction of form for further benefits. In the future, we hope to be able to connect the website with the machine learning model (currently they are functioning independently) as we were not able to do this due to time constraints.

Challenges we ran into

The main challenges we ran into were on the machine learning side. One challenge was that the original pre-trained model did not classify videos where humans are doing horizontal exercises (such as planks). To overcome this challenge, we optimized the model by feeding in more training images of people doing such exercises. Another challenge we ran into was using the model to provide real-time user feedback of corrections to their form and posture. We figured out by finding the slopes of various skeletal segments, as well as the angles between skeletal segments of the various important parts of the body for different exercises, we can provide custom and personalized feedback to users (btw the math for calculating the angles was hard to figure out lol).

Accomplishments that we're proud of

We are really proud of how we were able to use our basic machine learning experience to create a fully developing model and code for our purpose! Additionally, we are proud of how amazing our website turned out!

What we learned

We learned a lot about Python and machine learning, as well as some new skills and perks about HTML and CSS.

What's next for AI Train - The Perfect Fit

We want to develop a '1v1' system and a point system for competitions among friends. The point system would also be used to provide the user's progress. Currently, we have developed a website but we would also like to implement the model within an app for accessibility. Finally, we want to create the final bridge/connection between the model and the website.

Built With

Share this project: