• 

  leaderboard

• 

  landing page

• 

  homepage

• 

  profile

• 

  leaderboard

• 

  workout complete

• 

  correct rep

• 

  incorrect rep

RepQuest

What it does and why it matters

Staying active is hard when exercise feels like it needs a huge commitment, long gym sessions, and a fixed schedule. In reality, a lot of people just need a small push: a quick accessible challenge that fits in the middle of a busy day.

RepQuest was engineered around that idea. Our app sends users random workout challenges throughout the day, encouraging them to stay active through short, manageable bursts of exercise rather than long workouts. When a user opens a challenge, the app uses computer vision to analyze their movement in real time, count reps, and evaluate form quality. If the model detects in issue, it immediately displays it so the user can implement the fix and prevent long term injury as well as bad habits.

This matters because physical inactivity and obesity remain to be major health challenges in the US. Over 2 in 5 adults have obesity, 1-5 children are obese, and approximately a quarter of adults report no activity outside of work. To keep users motivated, RepQuest gamefiles this process with points, streaks, difficulty-based challenges, and leaderboards. Hitting daily point goals makes the experience feel less like a chore and more like a goal.

How we built it

ML (Ansh):

The first step to building our evaluator models was to find data. I searched online to find video data, but I couldn't find anything relevant. So I turned to recording my own data. I recorded about 20 videos for each variation of each exercise. Some of these variations would include mistakes done intentionally while others would be with perfect form. Currently, we support three common exercises: pushups, lunges, and squats. The model in question is an encoder-decoder architecture with a conv encoder and a transformer decoder. Our model takes in tensors with length 106 and 110 features, 99 of these being derived posed landmarks from Mediapipe, and the rest being calculated features, such as angles. After hours of testing, the evaluation models were finally accurate. Now we needed a way to detect reps. I initially decided to build a binary classifier that predicts if a rep has just been completed. After some testing and failure, I realized such an approach was not necessary at all. Instead, I switched my focus to using a rule based system. Reps follow a simple cycle: up -> down -> up. I took advantage of this and used angle thresholds to detect reps. This work with stunning accuracy—almost 97%, missing a few reps only. The only issue is that it sometimes randomly fires and counts reps that never happened. This isn't extremely common, and we will work to fix it in the future. Even though we only have three exercises, it is extremely easy to add more. We abstracted our entire pipeline into a universal class, meaning that all we need to do is record data and train—and boom, new exercise.

Frontend (Amogh):

Challenges

ML:

1. Evaluator model was initially extremely poor in performance, it only only improved once I added the calculated features to give the model more context.
2. Data was available for our use case, and so I needed to record them myself. This induces bias into our model, but I attempted to combat this by normalizing coordinates to ignore body size and keep it in proportion.
3. ML-based rep counter didn't work, and so I needed to switch to a simpler approach while maintaining high accuracy.

Frontend:

Accomplishments:

ML:

1. Realizing that the ml-based rep counter was just a case of over engineering, and that the simpler approach was just smarter here.
2. Getting the evaluator models to work after being broken for so long.
3. Abstracting the exercise creation pipeline to make adding new exercises very straightforward.

Learnings

ML:

I learned that sometimes, the best solution isn't the one that sounds the coolest, but the one that does the job the best. It's the solution that does what it needs to do without adding unnecessary fluff and possibilities for problems.

What's next

We plan to add:

• more exercises for a wide array of muscle groups
• actually deploy our application (its already on a landing page but still a few things need to be worked out from local to actual deployment)
• a friends system where you can see your friend's points, challenges, etc
• hostable challenges that allow users to create tournaments for who can amass the most points in a certain period of time or hold the longest streak
• polish UI and make it looker more professional and customizable

Built With

• mediapipe
• opencv
• python
• react
• react-native
• tensorflow