CreaseCoach: AI Lacrosse Assistant

Inspiration

We're a team of three UCSB club lacrosse players dedicated, both on and off the field, to ensuring that we win the WCLL championship for the 4th year in a row. During this hackathon, we decided to use our engineering and programming skills to build an AI-powered voice system that gives lacrosse goalies real-time coaching about players on the opposing team.

What it does

CreaseCoach analyzes live game footage and intelligently delivers audio callouts to the goalie through their headphones, providing critical intel like: “Number 24 Bastian approaching—lefty with a quick release, watch for the split to his strong hand.” It’s like having an experienced coach in your ear throughout the entire game, helping you anticipate threats before they develop.

The system features a web-based scouting interface where coaches can build scouting reports with detailed player profiles, including threat levels, shooting tendencies, and tactical notes. During gameplay, CreaseCoach uses parallel consensus detection to minimize false positives and smart cooldown timers to prevent audio spam, ensuring goalies only hear the most relevant callouts at the right moments.

How we built it

• Computer Vision Pipeline: Gemini 3 Flash Vision API for jersey number detection with parallel consensus (runs two API calls simultaneously and takes the intersection to reduce hallucinations)
• Video Processing: For demo purposes, live camera footage is replaced with an MP4 file, processed with OpenCV
• Natural Language Generation: Gemini 3 Flash LLM transforms player data into natural, conversational callouts
• Text-to-Speech: ElevenLabs API for dynamic, high-quality, low-latency voice synthesis
• Database: MongoDB Atlas for storing scouting reports, game sessions, and detection logs

Challenges we ran into

• Detection Accuracy: Finding the right balance between cost, speed, false positives, and false negatives was difficult. Early versions hallucinated jersey numbers, so we made the detection prompt more conservative with strict visibility rules. We added a consensus algorithm that compared the results of two parallel API calls to reduce model error and ultimately settled on the state-of-the-art Gemini 3 Flash Vision model for maximum performance at usable speed.
• Performance Optimization: Playback footage initially failed to render smoothly at real-time speed. We reworked parts of the pipeline and added frame skipping to prevent lag and ensure consistent real-time performance.

Accomplishments that we're proud of

• Built a complete end-to-end system from video input to audio output in a short timeframe
• Achieved real-time performance on 60fps video with low latency between detection and callout
• Created a consensus detection system that greatly reduced false positives
• Built an intuitive web interface that allows non-technical coaches to easily create scouting reports
• Successfully imported and tested with a real 23-player Northeastern Huskies scouting report

What we learned

• Full-Stack Integration: Connecting computer vision, NLP, cloud databases, and audio synthesis into a cohesive user experience
• Prompt Engineering: Making AI more cautious and reliable often means being explicit about what not to do

What's next for CreaseCoach: AI Lacrosse Assistant

• Live Camera Feed Support: Process real-time video from a camera operator or drone during actual games
• Secure Bluetooth Headset Connection: Connect the audio system to an in-helmet earpiece for in-game use
• Expansion to Other Sports: Adapt the system for hockey, soccer, water polo, and other goalie-based sports
• Post-Game Analytics: Generate detailed reports showing which players were most active and how the goalie responded
• Custom Voice Personas: Allow teams to use their actual coach’s voice for more authentic callouts (via ElevenLabs)

Built With

• elevenlabs
• fastapi
• google-cloud
• mongodb
• python