FitBuddy

• GIF

Inspiration

All of us in our group have at some point felt demotivated while exercising. We know that others feel this too. Exercising is something that everyone wants to be able to get into, but it takes willpower and effort to do so. The goal of FitBuddy is to help and fuel the motivation needed to get into and continue exercising.

What it does

Using pose estimation it creates values for specific nodes located around the body. Then trained on a dataset of up pushup and down pushup positions, it classifies whether you are in an up or down position. Cycling from the up position to the down position and back up causes one rep to be recorded. The user can set goals for how many reps they are trying to complete, and are rewarded when they complete that many reps.

How we built it

Google had a base extended pose classification google colab. We used that as our base, and then we used a Kaggle dataset of up and down position pushups, reformatted it, bootstrapped it, and finally used it in order to train our pushup classifier. Then we moved the google colab to a local IDE in order to try and circumvent the overheard that receiving a webcam stream in google colab would cause, as you are forced to only be able to interact with webcam streams through javascript. Finally after doing that we used Streamlit to host a website that showed the webcam feed with your pushups and your pushup counter. In addition, there was a text area in which you could enter in your goal reps. Upon achieving your goal reps a sound was played on the website.

Challenges we ran into

Step One: Finding Pose Estimation There were many, many challenges we ran into while doing this project. First was finding a way to do pose estimation, upon doing some research we started with using OpenPose, a repo created by CMU. Afterward we switched to using mediapipe, a google library for pose estimation.

Step Two: Finding Datasets After finding a module we could use, we needed to find datasets in order to train our prediction model. This wasn't an easy task, especially with formatting problems. We found data on Kaggle with a collection of many different pushup pictures in the up position and the down position. We then made a python script to correct format all of the dataset.

Step Three: Converting Video Input into Webcam Input Inherently, the colab file we built used video input in order to classify the total number of reps. To make our project and fulfill our goal, we wanted it to be in real-time. Converting from a webcam input into a video input was probably the longest and most time-consuming problem we found. Not only did we have to move off of Google Colab, as Google Colab inherently doesn't support webcam input with JavaScript, but we had to deal with the overhead that adding webcam input would cause. We found a way to get it to work in the end. This overhead and mainly speed, in general, became a reoccurring problem that we faced and will lead to our next problem.

Lastly, Front-End The problem with using a live webcam feed with an already inefficient and slow code is the fact that if done through a website, there is a ton of overheard between sending video and receiving the data. This means our already slow code became even worse and barely even worked. We fixed this a little by using a python library called Streamlit. This allowed us to create a website and significantly decrease the overhead. From there we added features to create our final project.

Accomplishments that we're proud of

We actually got our classifier to work. It can correctly count the number of reps you have completed. In addition, it plays a noise when you reach the goal rep. We overcame the overhead problem, which is something everyone on our team was extremely proud of.

What we learned

There's a ton of things we learned. First, we learned a ton about pose estimation and how it typically works. It was super interesting to see how it classified nodes and connections. Next, we learned how to work with Streamlit. In the process of transitioning to Streamlit, we also learned about fast API and flask and web socketing in general. All of this in general was an incredibly interesting project, and it was super informative overall.

What's next for FitBuddy

FitBuddy is super extendable. In a perfect world in which we had more time a ton more stuff could have been included. We could include an average time between reps, this could allow us to calculate things such as RPE, which is the amount of effort you put into your workout. Plus, since we were able to cause sound to play upon reaching a certain goal. We could reward users with points if they meet their certain pushup goals. Not only that but since our model is basically a classifier over whether a person is in an exercise-up position and an exercise-down position, this can be applied to a ton of other exercises, such as squats or pull-ups. If we had more time FitBuddy could truly be a Fitness Buddy., applying to many more exercises than just pushups.

GitHub Repo

GitHub Repo

Built With

• mediapipe
• python
• streamlit