-
-
Logo
-
Learning
-
Replay Analysis
-
Sessions
-
Stats
-
Coach
-
Controls
Inspiration
Traditional fighting-game training is an exhausting grind: you play 500 matches online, lose to gimmicks you don't understand, and hope you eventually improve. Static training modes only give you a non-reactive dummy, leaving you to figure out your own mistakes. In real combat sports like boxing, a human sparring partner actively reads your tells, capitalizes on your weaknesses, and forces you to adapt.
We built Kumite to bring that dynamic to video games: creating an AI sparring coach that isolates your flaws in real time and teaching you how to fix them between rounds.
What It Does
Kumite is an offline Tekken 8 sparring partner that reads the game's memory (read-only), plays against you as Kazuya via a virtual controller, and actively hunts down your gameplay flaws:
- Adapts in Real Time: It measures your habits round by round, detecting if you fail to block lows, break throws, or miss punish opportunities, and uses Thompson-sampling bandits to exploit the exact mistakes you keep making.
- Intelligent Voice Coaching: Between rounds, an AI coach powered by ElevenLabs gives you grounded feedback on what weaknesses were exploited and what to adjust.
- Dynamic Mastery & Re-testing: When you successfully correct a habit, the bot stops exploiting it, announces that the habit is fixed, and periodically re-tests you later to ensure the habit stuck under pressure.
- Continuous Personalization: Your player profile persists across sessions, and the internal model fine-tunes itself on your own play after every match.
How We Built It
Kumite was built in 24 hours across four specialized tracks. We froze our API contracts at hour zero, allowing three members to build against mock servers while integration ran on a single gaming laptop:
- Track A (Game & Learning Loop): Python
ReadProcessMemorypolling at ~1 kHz (2-frame input latency), feeding a reflex layer, Kazuya frame-exact combo replays, River online ML predictors with ADWIN drift detection, Bayesian Knowledge Tracing for habit mastery, and Thompson-sampling bandits for tactical exploits. - Track B (App & UI Overlay): An Electron, React, and Vite desktop app featuring an in-game HUD overlay, session tracking, mastery screens, and ElevenLabs voice output.
- Track C (Replay & Coach Service): A dedicated coaching service powered by
gpt-4.1-minithat delivers grounded round tips in under 1 second, complete with frame-accurate citations for post-match Q&A chat. - Track D (Analytics & Infrastructure): An analytics service tracking performance metrics, Tiger Data integration, mock API servers, and Sentry observability across all four services.
Challenges We Ran Into
- Keeping the Game Smooth: Running AI predictions inside a super-fast 1,000-times-per-second loop could have caused lag. We fixed this by moving data processing to background threads so the game stayed smooth
- ** Stopping False Fixes**: Early versions of the AI would mistakenly say a bad habit was "fixed" after just one good round. We fixed this with strict tracking rules, requiring players to prove consistency across multiple rounds before celebrating a fix
- Perfect Combos: The AI's attack combos kept failing halfway through. We fixed this by recording real human inputs and playing them back frame-by-frame.
* Memory Limits: FRunning the AI model and Tekken 8 at the same time took up too much VRAM. We solved this by fine-tuning only the top layers of the model, which saved memory.
Accomplishments That We're Proud Of
- Measurable In-Session Adaptation: Successfully creating an AI opponent that dynamically adjusts its playstyle to punish human tendencies and backs off once the player adapts.
- Honest Evaluation: Verifying model performance on held-out session datasets with balanced accuracy metrics rather than relying on qualitative guesswork.
What We Learned
- Contract-First Architecture: Freezing schemas on day one was crucial for enabling four developers to work simultaneously without needing access to the live game executable.
- Data-Driven Validation: Evaluating AI adaptation against majority baselines, balanced accuracy, and replayed session data was essential for proving actual learning rather than random variation.
What's Next
- Expanding Laya's classification capabilities to handle fine-grained movement (such as left vs. right sidesteps).
- Expanding the roster beyond Kazuya to support multiple bot characters.
- Enabling Laya-driven tactical decision-making and per-match automatic fine-tuning by default for all sessions.
Log in or sign up for Devpost to join the conversation.