Cloud9 AI Assistant Coach

Moneyball for Valorant: Real-time AI coaching via GRID & Gemini.

Comment

💡 Inspiration

Traditional sports were revolutionized by "Moneyball"—using data to find undervalued strategies and players. Esports generates even more granular data, but the game moves too fast for human coaches to process every variable in real-time.

We wanted to build an AI Assistant Coach that bridges the gap between raw data and winning decisions. We didn't just want a dashboard; we wanted a tireless strategist that understands the match context and can literally speak to the coach during the heat of battle.

🚀 What it does

Cloud9 AI Assistant Coach is a comprehensive analytics suite that combines live official esports data with multimodal generative AI.

  1. The Command Center (Live Dashboard): Visualizes real-time match scores, economy win rates, and objective control directly from the GRID Esports API.
  2. AI "Moneyball" Analyst: Uses Gemini 3.0 Pro to analyze deep player metrics (like First Death Impact and KAST consistency) to recommend "Buy/Extend" or "Sell/Trade" decisions.
  3. The Coach's Voice: Converts text-based insights into natural speech using Gemini 2.5 Flash TTS, allowing coaches to listen to briefings without taking their eyes off the game.
  4. Strategy Lab ("What If" Simulator): A predictive engine where coaches can ask natural language questions (e.g., "We are down 3 rounds on Ascent. Should we force buy?"). The AI uses the exact live match state to predict outcomes.
  5. Automated Match Review: Instantly generates rigorous, critical post-match meeting agendas focusing on macro mistakes and economy management.

⚙️ How we built it

  • Frontend: Built with React 19, Tailwind CSS, and Recharts for a responsive, futuristic "War Room" interface.
  • Data Pipeline: We integrated the GRID GraphQL API to fetch live Series, Match, and Roster data. We implemented a robust "Circuit Breaker" pattern that seamlessly switches to cached fallback data if the live feed is interrupted, ensuring the app is always demo-ready.
  • AI Engine: We utilized the Google GenAI SDK (@google/genai) to power our features:
    • Gemini 3.0 Pro: Used for complex reasoning, player evaluation, and the "Strategy Lab" prediction engine.
    • Gemini 2.5 Flash TTS: Used to generate low-latency, high-quality audio for the "Coach Voice" feature.

🚧 Challenges we ran into

  • Data Interpretation: GRID provides a massive amount of granular data. Inferring high-level narrative events (like "First Blood Impact") from raw stats required complex client-side logic.
  • Real-time Context: Injecting the exact state of a live match (map, score, specific round history) into the AI's context window without exceeding token limits or inducing hallucinations was a fine balance.
  • Audio Latency: Ensuring the Text-to-Speech felt responsive required optimizing how we buffered, decoded, and played back the raw PCM audio data from the Gemini API.

🏆 Accomplishments that we're proud of

  • The "Strategy Lab": Successfully creating a feature where the AI understands the context of a specific match to answer hypothetical questions. It feels like magic when the AI references the specific opponent and score in its answer.
  • Multimodal Experience: Moving beyond just text—having the application speak to the user creates a much more engaging experience.
  • Resilience: The application is incredibly robust. It handles API failures gracefully and provides a smooth user experience even in offline modes.

🧠 What we learned

  • Prompt Engineering for Persona: We learned how to prompt Gemini not just to be an assistant, but to adopt the persona of a critical, analytical Head Coach.
  • The Power of GraphQL: Efficiently querying only the exact nested data we needed from GRID was crucial for performance.
  • React 19 & GenAI: Combining the latest React hooks with streaming AI responses allows for highly interactive UI patterns.

🔮 What's next for Cloud9 AI Assistant Coach

  • Video Integration: Integrating Gemini 2.0's video understanding capabilities to analyze replay clips for mechanical errors alongside the statistical analysis.
  • Opponent Strat Prediction: Using historical data to predict enemy tendencies (e.g., "They fake A site 70% of the time on pistol rounds").
  • Mobile Companion App: A simplified interface for players to review their own AI-generated critiques between matches.
Share this project:

Updates