Gameday Companion

We are redefining the gameday experience with live AI chat, computer vision play breakdowns, and real-time player and team stats - giving fans instant insights and analysis as the action unfolds.

Comment

Gameday Companion

Inspiration 🏈

We were inspired by NFL NextGen Stats Powered by AWS and the revolutionary way they brought advanced analytics to football. Similar to the PrizePicks mobile experience, we wanted to bring gameday analytics to the palm of your hand.

As football fans, we've seen popular apps like Real, theScore, Sleeper, ESPN missing key features like:

  1. Live football data that we could actually chat with - imagine asking "How many yards does Lamar Jackson have right now?" or "What exciting game should I watch?" and getting an instant, accurate answer
  2. The ability to detect defensive coverages in real-time - many times we’ve wanted to uncover the strategy behind different formation

The combination of these desired features led us to build Gameday Companion: the ultimate fusion of play breakdowns, live data, and AI-powered analysis.

What it does 📱

Gameday Companion is a React Native mobile app that transforms how fans interact with live football. Our platform offers:

🤖 Live Gameday Chat : Ask our AI-powered chatbot anything about live NFL and college games - from game breakdowns to play-by-play details to fantasy football stats, all sourced from real-time ESPN data loaded through our custom Web Socket.

👁️ Computer Vision Play Analysis : Take a picture of your TV or football field during a play, and our app will:

  • Detect and tag all players on the field using a custom-trained YOLOv11 model
  • Identify pre-snap defensive coverage schemes (Cover 2 Man, Cover 4 Zone, etc.)
  • Map each player's position onto a virtual 2D football field

🏆 AI Defensive Coaching :

  • Get expert-level analysis of your chosen defensive formations by sending mapped play data to our specialized coaching AI.
  • Save and revisit analyzed plays at any time.

How we built it 🛠️

Frontend: React Native Mobile App

  • Framework : Expo for cross-platform compatibility
  • Features : Camera integration, real-time chat interface, interactive field visualization
  • Why : Can be used from your couch or the stands! Multi-platform solution

Computer Vision Backend: express.js API on Vercel

  • ML Pipeline : YOLOv11 model fine tuned on Roboflow for player detection
  • Core Algorithm : Custom coordinate mapping system that converts pixel coordinates to yard positions
  • Smart Analysis : Deterministic algorithms for line of scrimmage estimation and defensive coverage classification similar to AWS Next Gen stats.
  • Storage : Supabase for saving plays

Live Data API: Python Flask on Render

  • Data Source : Real-time ESPN scraping for NFL and college football.
  • AI Integration : Use LangChain Tool on OpenAI GPT requests to maintain fresh/accurate data.
  • Caching Strategy : Cache manager that optimizes data freshness and performance

Challenges we ran into 😤

Machine Learning Model Data Integration

The defensive coverage detection model required handling multiple data formats and schemas:

  • Creating a robust coordinate mapping system that works across different TV angles and field perspectives
  • Developing deterministic algorithms that could reliably classify complex defensive schemes
  • Handling edge cases where player positions might be ambiguous

Real-time Data Synchronization

Balancing data freshness with API rate limits while maintaining sub-second response times required multiple iterations of our caching strategy.

Accomplishments that we're proud of 🎉

🥇 First-Ever Integration: We are the first app to successfully connect an LLM with live-streaming sports data. This combination creates an entirely new category of sports analytics tools by answering open-ended questions from What close games are on now? to How is Lamar Jackson playing today?.

⚡ Real-Time Performance: Our caching system achieves sub-second response times for live data queries while maintaining response accuracy with LangChain tool calling.

🧠 Democratizing Play Analysis: Our coordinate mapping algorithm can accurately any football photo into precise field positions, opening up analysis previously only available to professional teams Now you can save plays, analyze coverages, and receive feedback from your couch or local football field!

What we learned 📚

  • Computer Vision Pipeline : The complexities of real-world object detection and coordinate transformation
  • Data Pipeline Design : How to build resilient systems that handle the unpredictability of live sports data
  • Algorithm Development : How to create deterministic solutions for inherently probabilistic problems (defensive coverage detection)

What's next 🔮

User Feedback

  • Improved UI/UX: Receive and iterate on user feedback on areas of improvement.

Technical Scaling

  • Chatbot Speedup: Open AI response times slow down the chat bot. Streaming response, caching system prompts, or hosting the LLM could help solve this issue.
  • Real-time Streaming: Move from image-based analysis to live video stream processing

Gameday Companion represents the future of sports technology - where artificial intelligence, computer vision, and real-time data converge to create experiences that were unimaginable just a few years ago. We're not just building an app; we're pioneering a new way to understand and interact with the game we love.

Built With

Share this project:

Updates