Shadow Sync

🏀 Shadow Sync

🌟 Inspiration

Every athlete dreams of performing like the greats — from Steph Curry’s perfect jumper to Serena Williams’ powerful serve. We wanted to make that dream measurable and personalized. Shadow Sync was born to help athletes see exactly how their movements compare to legends — using AI to bridge the gap between ambition and mastery.

🧠 What We Learned

We explored how to integrate AI vision models with LLM-based analysis to deliver detailed, actionable feedback. We gained hands-on experience in pose estimation, video data processing, and AI-based motion comparison — while also learning how to optimize inference speed for real-time user feedback.

🏗️ How We Built It

Shadow Sync brings together a full AI-powered pipeline:

• Frontend: Built with Next.js, TypeScript, and TailwindCSS for a smooth, interactive user experience.
• Backend: Implemented using FastAPI to handle video uploads, frame extraction, and inference requests.
• Pose Tracking: We used MediaPipe and OpenCV to detect body joints and generate keypoint data.
• Comparison Engine: Leveraging Google Gemini, we analyze pose vectors and movement dynamics between the athlete and a professional athlete’s recording. Gemini helps interpret subtle biomechanical differences and generate natural-language feedback.

⚙️ Challenges We Faced

Pose accuracy: Different lighting and camera setups caused inconsistent keypoint detection. Model integration: Combining MediaPipe outputs with Gemini’s analysis required careful formatting and prompt design. Performance: Optimizing the video processing pipeline to handle multiple frames quickly without losing accuracy.

🚀 What’s Next

We plan to enhance feedback personalization using Gemini’s multimodal capabilities, expand support to multiple sports, and introduce real-time side-by-side comparisons — making Shadow Sync a universal AI coach for athletes worldwide.

Built With

• fastapi
• gemini
• python
• react
• sqlalchemy
• typescript