FormCheck

Physical Therapy brought to the palm of your hand.

Comment

Inspiration

I was inspired to take on this project as I have struggled with chronic knee pain for the last 5 years of my life, with tendinitis in both knees. I have gone to physical therapy multiple times, and while it has been productive, someone cannot be there to constantly judge my form and give me feedback for exercises to prevent re-injury. Squatting is something in particular that has been challenging for me, which is where I was inspired to start this project. I wanted to build something real, that would benefit me and potentially others in everyday life beyond just this hackathon.

What it does

FormCheck is an AI fitness trainer that uses a smart bracelet and computer vision to analyze real time movements and gives constructive, personalized feedbacks on your workouts.

The user wears the smart bracelet and a camera which track movements and stability. After completing a set, the AI fitness trainer gives feedback on which parts of that set were good and what can be worked on. The user can also give the trainer points of interest to focus on, such as knee stability, and give feedback based on those provided comments.

How we built it

Hardware

  • Seeed Studio XIAO ESP32-S3(MPU)
  • HiLetgo GY-521(IMU)
  • AKZYTUE 3.7V 400mAh LiPo Battery
  • Logitech Brio 100 Webcam
  • Nylon Webbing

Software

  • Onshape
  • Python
  • VScode
  • Arduino
  • Google Gemini API
  • MediaPipe

Challenges we ran into

Hardware Challenges

  • Post print design issues
  • Physical port connection
  • Solder joints

Software Challenges

  • Rate limit on Free Tier Gemini
  • Knee Angle Tracking on Side Profile
  • Coach AI Misread Rep
  • Scoring Software Tuning ## Accomplishments that we're proud of
  • Built a fully working end-to-end pipeline, CV-> scoring-> wearable-> Ai Coaching
  • Designed a scoring system that's rule-based and explainable
  • Successfully integrated two competing signal sources (camera + wearable) ## What we learned Rule based scoring needs more than just intuition, and requires multiple trial runs and data in order to accurately tune.

What's next for FormCheck

Next on FormCheck is improving Quality of Life features, such as:

  • More Secure Lid
  • Waterproof
  • Simple Wristband Adjustment
  • Create Designated LiPo Containment
  • Support for additional exercises beyond squats
  • Native Phone App
  • On device local inference- privacy first approach for personal fitness.

Built With

Share this project:

Updates

Submission history