Imperfect Coach

Inspiration

Movement injuries happen because generic fitness apps can't see what your
body is actually doing. We wanted to build something that acts like a
digital physical therapist — catching dangerous form breakdowns before
they become injuries.

What it does

Imperfect Coach is an AI Biomechanics Lab that uses your webcam to analyze
 exercise form in real-time. It detects joint stress in your back and
knees, provides clinical-grade feedback, and autonomously prescribes
recovery protocols when it detects injury risk.

How we built it

 - Amazon Nova 2 Extended Thinking for step-by-step biomechanical reasoning
 - Bedrock AgentCore for autonomous multi-agent coordination (fitness coach
   ↔ recovery specialist)
 - TensorFlow.js + MediaPipe for 17-point pose detection
 - Multimodal analysis combining raw video frames with pose data
 - x402 protocol for agent-to-agent payments on Base/Avalanche/Solana
 - ag-ui patterns for user hint injection and human-in-the-loop
   authorization

Challenges we ran into

 - Getting Nova 2 to reason about physics and leverage rather than generic
   fitness advice
 - Synchronizing multimodal inputs (video + pose data) without latency
 - Implementing x402-style payments with real settlement across 3 chains
   simultaneously
 - Balancing Extended Thinking depth with real-time coaching responsiveness

Accomplishments that we're proud of

 - Clinical-grade injury detection for lumbar and knee stress
 - Autonomous rehab protocols triggered by detected biomechanical issues
 - Multi-chain x402 settlement working in production (Base, Avalanche,
   Solana)
 - Human-in-the-loop authorization — users approve AI prescriptions before
   activation
 - Live demo at imperfectcoach.netlify.app 
   (https://imperfectcoach.netlify.app)

What we learned

 - Extended Thinking models need structured prompts to show their reasoning
   chain
 - Multimodal vision requires careful frame selection to avoid overwhelming
   the model
 - ag-ui interaction patterns (hints, HITL) significantly improve user trust
    in AI decisions
 - x402 can be implemented with zero escrow using signature-based
   verification

What's next for Imperfect Coach

 - Phase 2/3: Decentralized agent discovery service (permissionless
   specialist marketplace)
 - More exercises: Squats, deadlifts, overhead press
 - Wearables integration: IMU data for 3D joint angle accuracy
 - Clinical partnerships: Validation studies with physical therapy clinics

Built With

• amazon
• nova
• react
• tensorflow
• typescript
• x402