It can be difficult to self-assess your form when weightlifting, and without assessment, improvement is unlikely. Finishing a workout is always satisfying, but we wondered if there was a way for us to know how well we performed. This brought about the idea of Smart Gains - an Android WearOS application for smartwatches to assess form through a multitude of metrics. The more we looked into the topic, we discovered that poor form weight training is a major driving factor for injuries and hospitalizations, and studies have indicated that weight training-related injuries are on a steep rise. Using bicep curls as a baseline, we figured that if we could harness the power of WearOS to understand and project the physics of these motions into form scores, the same theories could be applied to generalize the tool for a number of other exercises, including high-risk exercises such as skull-crushers, squats, and bench press.
What it does
SmartGains harnesses the power of smartwatch sensors through a WearOS app to measure various aspects of acceleration and tilt and translate this into genuine interpretations of muscle motion form. Specifically, we configured it to track the motion of a bicep curl and assign scores to each rep. When a "good" rep is detected, the number of reps completed increments on the watch face along with a circular progress bar. Ultimately, at the end of each set, you are notified that you have completed a full set of "good" reps via haptic feedback. On the watch face now, you find out your "grade" and how many of your reps were considered "good". A "good" rep is defined as one that fulfills the properties of smooth motion, controlled motion, full range of motion, and appropriate angles without exceeding any boundary thresholds that would be considered dangerous or ineffective.
How we built it
A significant portion of the project was figuring out the physics and vector algebra/calculus of the bicep curl motion. The movement of the watch in an arc means the whole coordinate system changes each instant, and so for any vectors that we were measuring, we couldn't just think about how that vector changes in a static space - we had to think about how the vector changes with respect to the new coordinate system at any given instant.
After this, we took several readings from the watch sensor of various types of motions in different directions and of different natures of smoothness or controlledness. We plotted all this data on a number of scatterplots and analyzed how the coordinate system of the watch corresponded to the theoretical calculations and what actual values would be relevant to extrapolate into thresholds. For example, we were able to match up our expectations for a particular sinusoidal function as a particular acceleration component to a graph that exhibited the appropriate properties and then manipulate that vector in our test run to see how the data changed, from which we could draw a connection between a misstep in form and an inconsistency in the function.
Then, it was a matter of writing Java code with Android WearOS to detect each rep and perform all sorts of analytics on the sensor data, which were defined by the metrics we discovered to be associated with particular form missteps. Detecting reps came down to more math related to sampling derivatives of the acceleration curve. In order to evaluate smoothness, we used a Harmonic Curve Fitter to perform a sinusoidal regression and computed the error between the data and the expected function for the given acceleration component. To compute tilt, we applied some operations upon the relevant component function to transform it into strictly an error function and integrated the function across the rep. For certain attributes, we had to calibrate the normalization ranges based on tons of bicep curl data. This was crucial because certain features have much wider ranges across which they exhibit themselves, but this has no bearing on whether those features should dominate in the overall score. After normalizing the features with calibrated ranges, we were able to construct an appropriately weighted total score for the set.
Challenges we ran into
The project as a whole was immensely challenging for our team's skillset. Given that we generally specialize in AR and ML, and none of us have ever developed a smartwatch application, developing for WearOS was a whole new experience that we had to spend a lot of time getting acquainted with. This process was made slower by the fact that we had to connect the smartwatch to the laptop via WiFi for development purposes. Furthermore, since the core of our project revolved around using sensor data, effectively only one person could develop at one time since we only had one smartwatch, and testing would not be possible on emulators (since they would not have physical sensors).
The other massive challenge was the physics of the situation. Since the sensors measure with the watch as the origin of the coordinate system, we were essentially dealing with a moving coordinate system that traveled in a rotational arc, and we had to figure out how the vector components were changing over time with respect to the changing coordinate system. Once we had a good understanding of how the vectors were meant to operate, we had to again tackle the situation with various potential form mistakes to visualize how the vectors would change in these circumstances.
Accomplishments that we're proud of
We're really happy that our theoretical calculations actually translated into useful results that mimicked reality, since initially, the physics seemed quite daunting. We're also really proud of ourselves for developing a full application with WearOS, which we'd never touched before. Also, we think it's super cool how we were able to harness acceleration data to have such accurate readings and reflections, not just for registering reps but also for interpreting the quality of their form.
What we learned
We learned about how some WearOS sensors work and how to develop for WearOS in general.
What's next for Smart Gains
There are a lot more things we can do with the application as we constructed it and the data we have been able to extract. Since each component of the score is derived from a particular form misstep, a simple examination of the data allows us to get detailed insights into what a particular individual might be commonly messing up. The problem is that having these detailed insights on a WearOS app would likely be clunky, so in the future, we would want to sync to a phone companion app that contains these insights and perhaps all historical workout grades. Furthermore, it wouldn't be too difficult to extend this to multiple other exercises such as skull crushers, bench press, and squats, so that would be one of the other next steps.