Heart Disease remains the no. 1 cause of death in the US, according to 2018 data. Heart disease accounts for approximately 13% of deaths in the US in 2018, causing 365,744 deaths. This is how it starts. Did you know? About 48 percent of adults have some form of cardiovascular disease and this is due to obesity. Cholesterol, high blood pressure, diabetes. These diseases may be genetic, but that's not the only factor causing them. People's lack of exercise is causing them to contract harmful diseases. However, we at FitBud have solved this problem.

What it does

FitBud is a life-changing web application that uses machine learning algorithms and large amounts of data to detect exercises, count repetitions, and estimate the number of calories burnt. First, the user makes an account through the login/register page. Then, if it's a new user, they will be directed to the new user form, where important data is gathered to create the most effective workout. Finally, they will be sent to the profile page, containing a personalized workout calendar, leaderboard, and a button that sends them to the video chat page. On the video chat page, the user's exercises are detected and everything is displayed accordingly. They are then given the option to end the session and view their stats.

Tools we used:

  • Firebase/React js to handle authentication which includes logging in and registering.
  • Tensorflow and PoseNet for exercise and repetitions
  • ReactJS for our UI and front-end/some backend logic
  • HTML and CSS to create and style the webpage
  • Kendoreact for a personalized exercise scheduler embedded within the web app(this helped us a lot)
  • Web-RTC for the video chat feature

How we made it

Our web application utilizes Tensorflow, specifically the PoseNet model, and the user's webcam camera, to handle pose estimation and exercise detection.

  • The PoseNet model gives us the key points of 17 different joints including each joint's position and confidence score. We used angles between different joints using these key points to detect when a user is performing a particular exercise
  • We used Cosine Similarity, a model which allows us to measure similarity between two non-zero vectors,
  • After normalizing the vectors and calculating the cosine similarity, we found the Euclidean Distance which takes into account the user's relative positioning in the rating calculations.
  • Finally, these calculations are made in real-time to output a final confidence score, between 1 and 100, that is used to determine what type of workout it is.

Challenges :

  • Communication was also a difficult task as all of us are from different time zones, so finding a time to meet up was often difficult.
  • Pose detection and exercise counting was very hard, but we solved this by utilizing a few of TensorFlow's advanced features.
  • Using python as a backend was very difficult because the back-end logic had some bugs. Due to this, we had to use React js to handle the Firebase code.
  • Finding the right API to do what we wanted was a major issue that we faced as some APIs do not work or do not have proper documentation.

What's next for FitBud

  • One of our plans is to implement an algorithm that matches users based on their preferred workouts. However, we were unable to execute this due to time constraints on our project.
  • We plan on syncing with fitness apps such as IOS and Android fitness data, MyFitness pal, Fitbit, and Nike run to track step count, running distance, and other vital stats

Research sources

Phillips, T. (2019, August 7). Exercise Classification with Machine Learning (Part I). Medium. https://towardsdatascience.com/exercise-classification-with-machine-learning-part-i-7cc336ef2e01 Soro, Andrea et al. “Recognition and Repetition Counting for ComplexPhysical Exercises with Deep Learning.” Sensors (Basel, Switzerland) vol. 19,3 714. 10 Feb. 2019, doi:10.3390/s19030714


A. Nagarkoti, R. Teotia, A. K. Mahale and P. K. Das, "Realtime Indoor Workout Analysis Using Machine Learning & Computer Vision," 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2019, pp. 1440-1443, doi: 10.1109/EMBC.2019.8856547.


Built With

Share this project: