QuickFit

Inspiration

About 6 months ago, our friend (Sahej) tore his ACL while playing basketball. As a result, he’s been faced with a difficult 1-year recovery process that has been exacerbated by the ongoing COVID-19 pandemic. As part of his recovery process, he has to do very specific, targeted exercises. For them to actually be effective, his exercises all must be done properly, but he has access to his physical therapist to critique him only twice a week. That inspired us to create QuickFit, the all-in-one at-home-Workout app and fitness critiquer.

What it does

QuickFit has multiple purposes, but its main purpose is to check and critique the form of specific exercises. Using their cameraphones, users record themselves in real-time while working out, and our app determines the correctness of their form. The app displays improvement in exercise form and tracks weight loss and reps per minute over long periods of time.

How we built it

We used ARKit and RealityKit to first attach an AR model onto a human person. Via this model, we found key landmarks on the human body which we used to get specific positions. Through linear algebra we were able to take the dot product of two landmark vectors and divide by their magnitude to eventually find the angle between the vectors/body parts. The angle and translational data allow us to track the form of exercises and calibrate our model to effectively differentiate between proper and improper forms.

We even had to look through some past studies on the mechanics of both correct and incorrect forms of multiple exercises, including but not limited to bicep curls, squat jumps, and tricep extensions to calibrate our model.

Challenges we ran into

For us, AR was particularly difficult to deal with since none of us had experience working with ARKit or RealityKit before this hackathon. We struggled initially to get the model to lock onto the person and even show up in the app. Also finding information from the various joints was difficult due to the lack of comprehensive documentation on getting values from the AR models. We also ran into lag issues due to the large amount of data being processed live, and as the list increases in size the time it takes to find the max and min also increases.

Accomplishments that we're proud of

A functioning app that people like me (Sahej) can actually use during their road to full recovery. Getting the 3D model to load in and get values, and calibrate the model to differentiate between proper and improper form. Reducing the lag of oncoming data.

What we learned

Through this app, we learned about many mathematical concepts that coincide with Augmented Reality which helped us tinker with an AR model. We learned to fit a 3D model onto a person to find the specific orientation and position of body parts ranging from the parent joint of the hip to shoulders and even hands. We also learned that it’s unhealthy to perform a bicep curl incorrectly multiple times, as my elbow is in a lot of pain as I am writing this.

What's next for QuickFit

We hope to partner with physical therapy clinics and gyms alike to expedite the future rollout of our app and would like for our app to be used by many businesses as they seek to benefit their consumers. A feature we are interested in adding is a system where PT providers can comment on their user’s via user videos, all done completely remotely,