FORMA

A computer vision and AI powered all-in-one Physical Therapy assistant web app

Comment

Inspiration

FORMA was born from the critical need to bridge the gap between clinical physical therapy and at-home rehabilitation. With rising healthcare costs and limited access to in-person therapy sessions, many patients struggle to maintain proper exercise form and consistency in their recovery journey. We were inspired to create a solution that democratizes access to professional-grade movement analysis while maintaining the crucial connection between healthcare providers and their patients. The platform addresses the challenge of ensuring patients perform exercises correctly without direct supervision, reducing injury risk and improving recovery outcomes.

What it does

FORMA is a comprehensive AI-powered physical therapy platform that provides real-time form analysis and personalized coaching for rehabilitation exercises. The platform uses advanced computer vision and MediaPipe pose detection to analyze movement patterns, offering instant feedback on exercise form through visual overlays and AI-generated coaching suggestions. For patients, it serves as a virtual PT coach that ensures safe, effective home exercises with features like emergency stop protocols, form scoring, and progress tracking. Healthcare providers gain powerful patient management tools, enabling them to monitor multiple patients remotely, track progress through detailed analytics, and maintain therapeutic relationships through integrated communication features. The dual-interface design adapts to both user types, providing simplified, safety-focused experiences for patients while offering comprehensive clinical tools for providers.

How we built it

We built FORMA using Next.js 14 and React for a responsive, performant web application. The pose detection system leverages MediaPipe's vision models for real-time skeletal tracking, processing video streams directly in the browser for privacy and low latency. We implemented a sophisticated biomechanical analysis engine that calculates joint angles, movement symmetry, and exercise-specific metrics. The AI coaching system integrates with GPT-5 through a custom Mastra agent architecture, providing contextual, personalized feedback based on movement patterns and exercise history. State management uses Zustand for efficient real-time updates, while the UI employs shadcn/ui components with a custom bubble-inspired design system featuring soft gradients and rounded interfaces. Authentication and data persistence utilize local storage with plans for cloud migration, ensuring HIPAA-compliant data handling for healthcare applications.

Challenges we ran into

Achieving accurate pose detection across varied lighting conditions and camera angles required extensive calibration and fallback systems. We implemented adaptive thresholds and confidence scoring to maintain reliability. Processing complex biomechanical calculations in real-time while maintaining 30fps video analysis pushed browser performance limits, leading us to optimize our algorithms and implement frame buffering strategies. Designing an interface that serves both healthcare professionals and patients with vastly different needs required careful consideration of information architecture and adaptive UI components. Ensuring the AI coaching remained medically accurate while being accessible to non-technical users involved creating multiple feedback layers with varying technical depth. Privacy concerns around video processing led us to implement entirely client-side analysis, avoiding any video transmission to servers.

Accomplishments that we're proud of

We successfully created a fully functional real-time movement analysis system that runs entirely in the browser, ensuring complete privacy while maintaining professional-grade accuracy. The platform's ability to detect and warn about potentially dangerous form issues in real-time could prevent countless injuries. Our dual-interface design elegantly serves both healthcare providers and patients without compromising either experience. The AI coaching system demonstrates impressive contextual awareness, adapting its feedback based on exercise history and detected patterns. We're particularly proud of the emergency safety protocols and the comprehensive progress tracking system that provides actionable insights for both users and providers. The visually appealing, accessibility-focused design makes complex biomechanical data understandable for everyday users.

What we learned

Working with healthcare workflows taught us the importance of maintaining human connections even in technology-mediated care. We learned that AI coaching must be carefully calibrated to avoid overwhelming users while still providing valuable guidance. The project reinforced the critical importance of privacy in healthcare applications, validating our decision for client-side processing. User testing revealed that visual feedback through overlays is often more effective than text-based instructions for movement correction. We also learned that building trust in AI-powered healthcare tools requires transparency about system limitations and clear safety protocols.

What's next for FORMA

Our roadmap includes expanding the exercise library to cover more rehabilitation protocols and specialized therapy programs. We plan to implement cloud synchronization with end-to-end encryption for secure multi-device access and provider collaboration. A mobile app with offline capabilities is another chance for greater accessibility. Advanced features like 3D movement visualization, predictive injury risk assessment, and personalized exercise prescription based on recovery patterns are also planned for development. Additionally, we plan to build a community platform where patients can share experiences and providers can contribute exercise protocols, creating a comprehensive ecosystem for physical therapy innovation.

Built With

Share this project:

Updates