Inspiration

Working out alone often means guessing if your form is correct, and hiring a personal trainer isn't accessible for everyone. My co-founder and I wanted to bridge this gap. We realized that modern smartphones have enough processing power to run complex computer vision models locally, so we decided to build a pocket personal trainer that actually watches you and gives real-time feedback.

What it does

Replix is a mobile fitness app that uses your phone's camera to analyze your workout form. It maps 33 3D skeletal landmarks on your body in real-time to automatically count your reps, correct your posture, and log your progress. Everything happens on-device, meaning zero lag and complete user privacy since no video feed is ever sent to a server.

How we built it

We built the cross-platform application using React Native and Expo, with Supabase handling our backend database and user authentication. The heavy lifting is done through custom Expo native modules we engineered to integrate Google MediaPipe Tasks Vision and CameraX. This architecture allows us to run complex pose estimation models directly on the mobile hardware.

Challenges we ran into

Our biggest hurdle was getting real-time computer vision to run smoothly on a mobile device without dropping frames or instantly draining the battery. Bridging native MediaPipe tracking libraries into a React Native environment required writing custom native code, which was a steep learning curve. We also spent a lot of time fine-tuning the rep-counting logic and handling dynamic screen orientation controls without interrupting the active tracking session.

Accomplishments that we're proud of

We are incredibly proud of achieving low-latency, on-device ML tracking within a React Native ecosystem. Getting the 3D skeletal mesh to overlay perfectly onto the live camera feed was a massive technical win. We're also proud of taking the project from a rough prototype to a polished application under our Skortan organization.

What we learned

We learned how to break out of the standard Expo managed workflow to write native code when performance is critical. We also gained deep insights into on-device machine learning constraints, managing memory during live camera feeds, and optimizing real-time state updates.

What's next for Replix

Since we are focusing on the Next Gen Award to showcase our open-source repo and core tech, our immediate next step is expanding our supported workout library. With our subscription testing already completed, our final steps are wrapping up internal testing and completing the official submission processes for the App Store and Google Play Store.

Built With

Share this project:

Updates

Submission history