Better Moves

Inspiration

Are you new to your weightlifting and not sure where to start when it comes to form? A fitness veteran with a nagging injury? Regardless of where you land in your fitness journey, Better Moves is your personal resource to offer real time video analysis of human movement, suggest areas of improvement, and answer questions based on your personal movement analysis. With rising prices of health care and the growing need to prioritize health, Better Moves gives the ability to use objective data to perform better, avoid future injuries, tailor recommendations based on personal needs, and is an always available resource for questions based on an individual’s personal needs through the ChatBot.

What it does

Better Moves solves the guess work in working out. Rather than googling answers that have questionable accuracy. Better Moves gives you personalized analysis of your movements by showing where your form is lacking, recommendations to improve, and has a ChatBot function for real time Q&A.

How we built it

Better Moves is built using OpenCV and MPII to recognize points within the human body (knee, shoulder, ankles, etc). The points are used as datasets and analyzes for positive or negative posture during specific movements including deadlifts, squats, and more. From there, trained ridge regression for predicting posture, potential areas of improvement, areas of weakness, current or future injury, and gives feedback based on said analysis. Pinecone was used as the vector database that the LangChain Chatbot used to reference for questions and answers.

Download pose_iter_160000.caffemodel online [link(http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/mpi/pose_iter_160000.caffemodel) and put it in pose/mpi/pose_iter_160000.caffemodel

Run the jupyter notebook and enjoy! Below is an analysis on how this project was created.

This project analyzes videos/images of the deadlift (one of the most fundamental weightlifting exercises) and scores the posture of the person performing the deadlift from a range of 0 to 1.

This process can be simplified into a few main parts:

Analyze videos/gifs frame by frame (for high fps videos we can analyze every X frames since frames will be very repetitive and tune X accordingly to satisfy our needs). We will use the OpenCV object VideoWriter. For each frame we score the exercise posture. First we feed our image into an open source OpenPose keypoint detection model for human pose detection (this model has been pre-trained with ~40 000 examples using the MPII dataset). This gives us locations of specific joints (right shoulder, left elbow, etc) and then using the locations of these joints we can feed this data to a classification model to predict whether the performer of the exercise is in correct posture or not.

Attached are some example images are included at this stage of our pipeline.

Our classification step involves using a machine learning model to classify the posture of the individual performing the deadlift. Various models were experimented with such as Random Forest, Elastic Net, Lasso, Ridge, and Logistic Regression (and ensemble learning methods) but Ridge seemed to perform best on the given dataset.

The final step involves taking each posture score and printing this information onto the frame of the video (along with the time taken to classify the frame) and write each frame to our output VideoWriter object. Once all desired frames have been scored, we release our VideoWriter object.

Built using pre-trained weights for OpenPose keypoint detection using the MPII pose estimation dataset (see link for OpenPose example and other computer vision examples), Python, OpenCV, Scikit-Learn, Jupyter Notebook.

Challenges we ran into

The main challange we faced was exploring the vendors, understanding the tech stack and how we wanted to utilize it in order to build. We also ran across a couple bugs and accuracy issues that were a challenge to fix such as version errors and incorrect hyperparameter tuning.

Accomplishments that we're proud of

We were exceptionsally proud that we were able to use a ML score to get a response from the ChatBot that would improve fitness performance. On top of that, being able to use Pinecone in a way to solve accuracy issues that many fitness professionals struggle with!

What we learned

As hackathon newbies, we have definitely learned a ton about this process in general. On top of that, neither of us have worked with Vector Databases previously so this was a massive learning curve to figure out the use case and build the application around it.

What's next for Better Moves

Right now, Better Moves is set up to analyze a small amount of human movement. Our next steps include building the Vector Database around use cases for all different movements and exercises to expant the domain of what problems can be solved!

Overall we are able to analyze our deadlifting videos with some help with computer vision. The next step in this project would be expanding the data set for posture classification (step 4). The main challenge is finding a large enough data set to build a model that can generalize well and avoid overfitting to the small data set. However if a sufficient amount of exercise posture data is aggregated, our model can become extremely useful for fitness enthusiasts.

Built With

• computer-vision
• elastic-net
• jupyter-notebook
• keypoint-detection
• logistic-regression
• machine-learning
• mpii
• mpii-database
• open-pose
• openai
• opencv
• pinecone
• python
• scikit-learn