Ai Workout Assistant

Imagine a personal trainer in your pocket! Our AI-powered web app uses TensorFlow.js and MoveNet to give real-time workout guidance, improving form and boosting fitness anytime, anywhere.

Comment

Inspiration

The project was inspired by the idea of creating a web-based workout assistant that leverages TensorFlow.js to provide real-time pose analysis. The goal was to build a virtual fitness coach that could offer instant feedback and guidance, making workouts more effective and engaging for users.

What it does

The AI Workout Assistant uses MoveNet via TensorFlow.js to track and analyze a user's body pose in real-time through their camera feed. It displays a simplified skeleton overlay, evaluates posture and joint angles, and provides immediate visual and audio feedback (e.g., "Perfect squat!"). The application includes a library of guided exercises, automatically tracks reps and sets, and visualizes progress dynamically on a right-hand panel with a circular progress bar. It also color-codes the skeleton based on form (green for correct, red for incorrect) and aims to provide analytics and personalized workout suggestions.

How we built it

The application is built as a single-page application using React for the user interface. Tailwind CSS is used for styling, ensuring a modern and clean design. The core pose detection functionality is powered by TensorFlow.js, specifically utilizing the MoveNet model. Custom React hooks (useWorkout) manage the workout state and logic, while utility classes (PoseDetector, ExerciseAnalyzer) handle the TensorFlow.js integration and exercise-specific analysis. The Web Speech API is used for audio feedback. Vite serves as the build tool for the project

Challenges we ran into

During development, we encountered several challenges. Initially, there were issues with the camera feed appearing black, which required careful handling of camera access permissions and video stream initialization. A significant challenge was an error related to passing incorrect data types to the TensorFlow.js detectPoses function, which was resolved by ensuring the video element was correctly provided. We also faced the problem of repetitive "Cannot detect full body pose" messages flooding the screen, which was addressed by implementing debouncing mechanisms for feedback messages to improve the user experience.

Accomplishments that we're proud of

We are proud of successfully integrating TensorFlow.js MoveNet for real-time, in-browser pose detection, which is the core of the application. Developing a robust exercise analysis system that provides specific and actionable feedback to users is another key accomplishment. We're also pleased with the dynamic and interactive user interface, featuring real-time progress tracking, a visual skeleton overlay, and a circular progress bar, all contributing to a motivating workout experience.

What we learned

Through this project, we gained valuable experience in working with browser-based machine learning using TensorFlow.js and handling real-time video streams and canvas rendering. We learned the importance of implementing effective debouncing and feedback mechanisms to create a smooth and non-intrusive user experience. Additionally, we deepened our understanding of structuring complex React applications with custom hooks and utility classes for managing intricate logic, and the nuances of debugging issues related to camera access and ML model inputs.

What's next for Ai Workout Assistant

For the AI Workout Assistant, the next steps involve expanding the exercise library with more diverse workouts and refining the dynamic workout sequence and progression system to allow for more customization. We plan to fully implement the analytics and progression tracking features, including visual charts and estimated calorie burn. Exploring further AI integration, possibly with Gemini, for more advanced personalized workout suggestions is also on the roadmap. Finally, considering optional social or community features like group challenges or leaderboards could enhance user engagement.

Built With

Share this project:

Updates