MotionBuddy

Inspiration

Many people want to learn new physical skills but may not have access to a personal trainer, dance instructor, or yoga teacher. Watching a video is helpful, but it cannot tell learners whether they are performing a movement correctly or guide them when they make mistakes. This inspired us to build MotionBuddy, an AI-powered movement companion that aims to make learning physical movements more interactive, personalized, and accessible through a TV-based experience.

What it does

MotionBuddy begins with an AI-guided squat coach that helps users follow a reference demonstration while practicing the same movement. Using a webcam, pose estimation, repetition counting, and spoken feedback, the prototype can detect certain posture issues and guide users toward better form.

However, our vision goes far beyond fitness exercises. In the future, MotionBuddy aims to support dance learning, yoga poses, stretching, fitness routines, and many other physical skills. Users could follow a reference video, practice the movement themselves, and receive feedback on how their body position compares with the demonstration.

Our long-term goal is to transform passive video watching into interactive, movement-based learning—helping people practice at home at their own pace without always needing an instructor physically present.

How we built it

We developed the movement-tracking prototype in Python using OpenCV for video processing, MediaPipe Pose Landmarker for body landmark detection, and pyttsx3 for spoken feedback. We implemented squat repetition counting and basic posture checks using joint angles and torso alignment.

We are developing the TV interface in Android Studio using Kotlin and XML layouts, with a focus on a simple, accessible experience suitable for a large screen.

Challenges we ran into

One major challenge was detecting body landmarks reliably and identifying different stages of a squat. We also needed to tune movement thresholds and feedback timing to reduce incorrect repetition counts and repetitive voice instructions.

Another challenge was connecting a working Python prototype with an Android TV application. Since the two environments use different technologies, integrating the movement analysis into the TV experience remains an important engineering task.

Accomplishments that we're proud of

• Built a working AI movement-tracking prototype: We developed a Python-based prototype that uses webcam input and pose estimation to track body movements during squats.
• Implemented real-time feedback: Our prototype can count squat repetitions, analyze knee angles, identify certain posture issues, and provide spoken guidance to help users improve their form.
• Created a reference-based workout experience: We prepared a reference squat video and are working toward comparing the user's movements with the demonstration to make practice more interactive.
• Started building a TV-friendly interface: We developed the initial MotionBuddy interface in Android Studio using Kotlin and XML, designed around a simple workout-selection experience for a television screen.
• Combined multiple technologies into one project: We brought together computer vision, pose estimation, video processing, movement analysis, and text-to-speech feedback to address a real-world fitness challenge.
• Developed a vision beyond fitness: We envision expanding MotionBuddy beyond exercise into dance, yoga, stretching, and other physical skills, helping people learn movements by following demonstrations and receiving personalized feedback.
• Learned by solving real engineering challenges: We worked through pose-detection issues, movement thresholds, voice-feedback timing, and the challenge of connecting a Python prototype with an Android TV application. We're proud that MotionBuddy has grown from an idea into a working movement-coaching prototype, with the potential to make guided physical learning more accessible and engaging.

What we learned

This project helped us explore computer vision, pose estimation, joint-angle calculations, real-time feedback, and text-to-speech integration. We also learned that creating a useful AI product requires more than a working model: it requires thoughtful user experience design, reliable feedback, and careful testing across platforms.

What's next for MotionBuddy

Our next steps are to integrate the movement-analysis prototype with the TV interface, improve reference-versus-user movement comparison, and test the experience on the intended Fire TV platform.

Beyond the initial squat coach, we envision expanding MotionBuddy into a flexible movement-learning platform for dance, yoga, stretching, sports drills, and other activities. Over time, we aim to support different reference movements and provide personalized feedback that helps learners improve through practice.

MotionBuddy's vision: Watch it. Try it. Learn it. Move with confidence.

Built With

• android-studio
• github
• kotlin
• mediapipe
• opencv
• pose
• python
• text-to-speech
• video
• xml