Slugger Sensei: AI Baseball Coach

Inspiration

Baseball is a game of precision, timing, and technique, yet not everyone has access to top-tier coaching. Whether you’re a beginner learning the fundamentals or an advanced player refining your swing, personalized feedback can make all the difference. We wanted to bridge the gap between professional coaching and everyday players by creating an interactive and engaging approach to training. By tracking key performance metrics such as launch angle, exit velocity, bat speed, and swing efficiency, we provide players and coaches with real-time data to fine-tune mechanics. Through interactive visualizations, performance benchmarks, and hands-on coaching drills, Slugger Sensei makes baseball training both insightful and engaging.

What it does

Slugger Sensei is an AI-powered baseball coach that analyzes a player’s swing mechanics through video input and provides instant feedback to improve their hitting performance. Using computer vision and machine learning, it tracks key metrics such as bat speed, stance, hand positioning, follow-through, launch angle, and exit velocity. The system not only measures these metrics but also provides insightful data on a player's performance progression over time. Players receive detailed breakdowns of their launch angle and exit velocity, pinpointing exact areas for improvement. The AI assistant offers personalized coaching recommendations and explanations based on the analysis, ensuring targeted and effective skill development. Additionally, users can store and track their progress, compare swings to professional athletes, and receive tailored drills to refine their game.

How we built it

We leveraged computer vision models trained on thousands of baseball swings to detect key movement patterns. Using OpenCV and MediaPipe, we tracked skeletal motion, while TensorFlow powered our AI models to analyze performance metrics. To enhance tracking accuracy, we implemented a method for extracting ball coordinates using OpenCV & HSV filtering, with plans to switch to YOLOv8 for improved robustness against varying lighting conditions.

For launch angle and exit velocity calculations, we used projectile motion physics principles:

Launch Angle (θ) is derived using a quadratic equation fit to the ball’s trajectory and taking its derivative.
Exit Velocity is computed based on the ball’s movement across frames, converting pixel displacement into mph.

The backend, built with FastAPI, handles video processing and model inference efficiently, and is deployed on Google Cloud Run for scalability. The frontend provides an intuitive dashboard where users can upload videos, view analytics, and receive AI-generated coaching feedback.

Challenges we ran into

One of the biggest challenges was ensuring real-time feedback while processing high-resolution video. Optimizing model inference speed without compromising accuracy was tricky. Additionally, fine-tuning the AI to differentiate between small yet significant differences in swing mechanics required extensive training data and validation. Integrating launch angle and exit velocity tracking added another layer of complexity, as lighting and camera angles affected ball detection. Lastly, deploying the system on Cloud Run presented hurdles in managing video processing workloads efficiently.

Accomplishments that we’re proud of:

Successfully implemented real-time AI-driven swing analysis that provides actionable feedback.
Optimized video processing to run smoothly on Google Cloud Run, ensuring scalability.
Developed an intuitive user interface that makes AI coaching accessible to players of all skill levels.
Integrated launch angle and exit velocity calculations to evaluate a batter’s home run potential.
Created a modular system that allows for future expansion into other sports and player roles.

What we learned

This project reinforced the importance of efficient model deployment and optimization, especially when handling computationally expensive tasks like video processing. We also learned how to fine-tune AI models for motion analysis and integrate computer vision pipelines into a production-ready environment. Additionally, our research into launch angle and exit velocity statistics based on MLB data helped us refine our tracking methodology and establish benchmarks for player performance. Lastly, we gained a deeper appreciation for user experience design, ensuring that AI-generated insights are presented in an easy-to-understand manner for athletes.

What’s next for Slugger Sensei: AI Baseball Coach

We plan to expand Slugger Sensei’s capabilities by incorporating pitch recognition analysis, allowing batters to train against different pitch types. Additionally, we aim to develop a mobile app for on-the-go swing analysis and integrate real-time feedback via AR overlays. To accommodate both batters and pitchers, we propose a user role selection feature that dynamically switches between swing tracking and gesture-based pitching analysis. Beyond baseball, we see potential in applying this technology to other sports that rely on biomechanics, such as golf and tennis.