AI Hitting Coach

Imagine having a baseball coach analyzing your swing in real time. Real-time voice feedback to perfect your swing instantly. Swing smarter. Train faster. Become the hitter you were meant to be.

Comment 1

Inspiration

Baseball players at all levels struggle with perfecting their swing, often relying on expensive coaching sessions or video analysis that lacks real-time feedback. We were inspired to build an AI-powered hitting coach that can provide instant, dynamic feedback using computer vision and AI voice coaching—just like having a professional coach with you at all times.

What it does

AI Hitting Coach analyzes a player’s swing in real-time using MoveNet (TensorFlow.js) for body tracking and an AI Agent (ElevenLabs) for instant voice coaching. The system detects:

  • Swing mechanics and positioning
  • Common errors like dropping hands or shifting too early
  • Corrects mistakes with AI-generated real-time voice coaching

With AI-powered feedback, players can get immediate corrections instead of waiting for post-game analysis.

How we built it

We combined multiple technologies to bring real-time swing analysis to life:

  • Frontend: JavaScript, HTML, CSS, TensorFlow.js (MoveNet), WebSockets
  • Backend: Node.js, Express, Socket.io
  • AI: ElevenLabs AI Agents for natural voice coaching
  • Real-time Data Processing: MoveNet detects swing mechanics, and WebSockets handle low-latency AI-generated coaching responses

The system tracks swing movements, identifies errors, and speaks personalized feedback instantly.

Challenges we ran into

  1. Swing Detection Sensitivity – Initially, our model detected false positives from simple movements, requiring fine-tuning to only detect real swings.
  2. WebSocket Stability – Ensuring real-time communication between the frontend and ElevenLabs AI Agent required persistent WebSocket handling.
  3. AI Response Timing – Generating dynamic feedback without latency was crucial for real-time coaching.

Accomplishments that we're proud of

✅ Successfully integrated real-time AI voice coaching using ElevenLabs AI Agents
✅ Built a fully functional, hands-free swing trainer that provides instant spoken corrections
✅ Created an interactive, data-driven experience that mimics a real hitting coach
✅ Overcame technical challenges to ensure accurate and fast feedback

What we learned

🔹 Real-time AI interactions are challenging – Optimizing latency and response times was a key lesson.
🔹 Motion tracking requires fine-tuning – Raw pose data isn't enough; context-aware filtering is needed.
🔹 WebSockets provide a huge advantage – Moving from API polling to persistent WebSocket connections dramatically improved speed.
🔹 AI feedback needs natural variations – We learned how to train AI Agents to provide dynamic, non-repetitive responses for a better coaching experience.

What's next for AI Hitting Coach

🚀 Expand AI Knowledge – Train the AI Agent to detect more swing issues and provide customized training plans.
🎯 Multi-Player Mode – Enable teams to use the system for group swing analysis.
📊 Data-Driven Performance Tracking – Store swing data over time to provide detailed progress reports.
📲 Mobile App Integration – Bring AI Hitting Coach to smartphones for on-the-go training.

⚾ AI Hitting Coach is just getting started. The future of baseball training is here!

Built With

Share this project:

Updates