CS2 AI Coach Assistant: The Story

💡 The Spark of Inspiration

As competitive players, we realized that while Counter-Strike is a game of mechanical skill, it is equally a game of information management. In the heat of a 1v2 clutch, players often suffer from "tunnel vision," forgetting to check their utility or track the enemy's economy. We were inspired to build a "Battle Buddy" that uses the latest multimodal AI to act as a tactical observer, providing the kind of oversight usually reserved for professional coaches sitting behind a player at a Major.

🛠️ How We Built It

The project is a symphony of local data scraping and cloud-based intelligence. We structured the application into three core pillars:

The Sensory Layer (GSI & Vision): We utilized Counter-Strike's Game State Integration (GSI) to extract real-time JSON payloads. To supplement this "raw data," we implemented a screen capture system using mss and OpenCV, allowing the AI to "see" what the player sees.
The Nervous System (FastAPI & PyQt6): We built a FastAPI backend to handle asynchronous requests. This allows the coach to remain responsive even when the game is at 400+ FPS. The PyQt6 frontend serves as the command center, managing settings and displaying live status.
The Brain (Gemini 2.0 Flash): We chose Gemini 2.0 Flash for its industry-leading speed and multimodal capabilities. It doesn't just read the score; it analyzes the visual "vibe" of the round to offer advice like "They’re stacking A, rotate now!"

🧪 Technical Deep Dive

A major part of our logic involves calculating the Team Buy Potential. We don't just look at one player; we look at the collective economy to decide if a "Force Buy" is statistically viable.

The probability of winning a round $P(W)$ based on equipment value $V$ can be modeled as:

$$P(W) \approx \frac{V_{team}}{V_{team} + V_{enemy}}$$

Our agent_brain.py uses these variables to craft advice that isn't just "guesswork," but grounded in the current state of the match.

🚧 Challenges We Faced

The "Vocal" Latency Gap: Converting Speech-to-Text (STT), sending it to Gemini, and then running Text-to-Speech (TTS) can create a "laggy" conversation. We optimized this by using a threaded stt_listener.py and the lightweight gTTS library to ensure the coach's voice doesn't arrive after the round is already over.
Data Synchronization: Ensuring the MongoDB match history stayed in sync with live GSI events without causing memory leaks or database locks was a significant hurdle in the database.py implementation.
Multimodal Prompting: Designing prompts that tell the AI to ignore the player's weapon skins but focus on the "Kill Feed" and "Radar" required dozens of iterations.

📚 Lessons Learned

This project was a masterclass in Asynchronous Programming. We learned how to manage multiple "lifecycles"—the game's lifecycle, the API's heartbeat, and the user's voice commands—simultaneously. Most importantly, we proved that AI isn't just for chatbots; it’s a tool that can be integrated into high-performance, real-time environments like gaming.