Lifter demonstrating correct Deadlift form
Correct Deadlift form with body part tracking through Machine Vision model
As regular fellow lifters, we have noticed that many of our friends and aspiring new gym-goers tended to "go too hard" at the gym, often compromising their safeties or even injuring themselves as a result of lifting weights with poor form. Nevertheless, we understand that it can also be intimidating to approach other people at the gym, even if it is to simply ask for advice. As a result, we decided to build FormCheck, a comprehensive solution to help people raise awareness and improve their form while working out, while also pointing out the various potential weaknesses through a lifter's body movement.
What It Does
FormCheck is a web application that allows users to upload a video of themselves performing an exercise (e.g. squat, bench press, deadlift). FormCheck then analyzes the video using machine learning and image recognition, and produces an analysis of the lifter's form during the movement, as well as the lifter's potential weak points throughout the movement. A conclusion of the lifter's form during the movement is then shown to the user, and several links covering various exercises to help address these weak points are then generated and presented to the user.
How We Built It
We began by splitting into two groups, with two of us focused on front-end web design and development and the other two on back-end programming. As such, the back-end programmers began by working on a machine vision model and later focused on building the app's features and the data pipeline between Google Cloud's Compute Engine, Azure Storage, and our front-end website. Moreover, they focused on collecting data to serve as a testing sample and started to compare the results given by the model with proper workout techniques.
Challenges We Ran Into
We ran into several challenges when building our project. Firstly, several frameworks that we had decided to use contained extremely unclear installation instructions, which significantly delayed our progress in the project. Moreover, we discovered that our computers were not powerful enough to effectively run our machine vision model, which hence pushed us to learn and utilize Google Cloud Platform's compute engine for our back-end processes. Furthermore, due to our diverse skill sets and differing technical experiences, some of us were not acquainted with the various technologies that we decided to utilize. This caused delays in bringing all parts of our project together as it took time for each of us to understand the thought process of someone else's programming. Towards the end, we also struggled with combining the various platforms together and found it difficult to combine our modular parts to become compatible with each other.
Accomplishments That We're Proud Of
We felt very accomplished that we were able to implement an effective machine vision model to identify the body parts of a person and analyze their movements while working out. Furthermore, we felt proud of our ability to effectively plot the points identifying a person's body parts during movement, as well as the data pipeline that we built; the latter enabled us to effectively communicate between our front-end web app, Azure Storage, and our VM using Google's Compute Engine.
What We Learned
Our front-end developers furthered their skills in JQuery, Bootstrap, and Flask. Our back-end developers gained experience with both Google Compute Engine and VM management, as well as Microsoft Azure's Blob Storage system. Furthermore, we all learned a lot more about creating image processing models using machine learning. Finally, as we all came from various different technical backgrounds, we learned to adjust our work styles to accommodate each other's different skill sets and experiences.
What's Next For FormCheck
We want to build FormCheck into a simple and essential mobile/web app that's part of the daily experience for both aspiring gym-goers and professional athletes. Through FormCheck, we'd like to raise awareness for the importance of proper form in the gym, with the main mission of encouraging users to lead a healthy lifestyle by learning to exercise properly. Moreover, we'd also like to help professional athletes perfect their craft by aiding them in identifying and remediating their weaknesses.