-
-
Landing Page
-
Summary Page
Inspiration
We're passionate about fitness and physical therapy and want to enable everyone to take charge of their own physical health. The main difficulty with getting started is the lack of knowledge, confidence, or mentorship. Not everyone has a fitness guru as a friend that can guide them through their fitness journey, and it can be difficult to improve without frequent feedback. For physical therapists, it can be tiresome to monitor patients' progress due to all the paperwork that is required. In addition, it's difficult to accurately monitor patients remotely through mediums such as video-call.
What it does
Our solution is PhysioDash, a web application that automatically provides a summary of your home workout progress through your webcam. This application can be used to allow its users to easily track their workout progress or enable physical therapists to monitor their patients’ recovery and performance remotely.
How we built it
We built the front-end with React in order to allow the user to record and save their exercise session through webcam. The video is then fed into a pose classification machine learning model developed by MediaPipe on a Colab notebook, which comes equipped with a rep-counting feature. We trained this model on images from the UCF101 - Action Recognition Data Set of various exercises such as wall push ups and squats.
Challenges we ran into
We would have liked to run the pose classification model directly on our Flask backend as opposed to manually inputting video into the Colab notebook, but there were problems we were not able to solve in time such as getting MediaPipe’s pose classification model to work outside of the Colab environment. This may have been due to dependency issues, which we may be able to solve with Docker in the future. Another challenge we faced was the fact that the webcam video downloaded from the web is in the WebM format, which is incompatible with the MediaPipe model. We temporarily addressed this by manually converting our video to the appropriate format, but we will automate this in the future.
Accomplishments that we're proud of
We developed our first-ever tangible concept of this idea, which has given us much better insight on what we can improve and what challenges are associated with developing this type of application. We're proud of the front-end work we were able to get done in addition to getting the MediaPipe pose classification model working with our data.
What we learned
We learned about the difficulties of deploying a machine learning model on a web application. In addition, we gained a basic understanding of Flask, a Python micro web framework that can be useful for integrating machine learning models on web applications. We also gained knowledge and practice of how JavaScript gathers media data, such as video, for web apps.
What's next for PhysioDash
In the future, we will include new performance metrics such as exercise rep quality, average rep speed, and exercise time (time spent purely exercising). To improve the user experience, we will store the machine learning model in the Flask backend and automatically input the webcam video input into it in order to streamline the process of getting a workout summary. We also hope to allow our web app to provide real-time feedback to allow the user to maximize their workout's effectiveness.
Log in or sign up for Devpost to join the conversation.