ChessTactics - Voice Chess Coach

• 

  Tactical puzzle mode with back rank mate challenge and real-time coach feedback

• 

  Main dashboard with hands-free voice input, board view, and live audio coaching

• 

  Tactical puzzle session with real-time audio coach guidance and on-demand Master AI Insight.

• 

  Blindfold training mode with audio piece status recitation and a 3-second peek verification.

Inspiration

Playing chess without looking at the board (blindfold chess) is one of the most effective ways to build deep board vision, spatial memory, and tactical foresight. However, practicing it alone is notoriously frustrating. We wanted to create an intuitive, hands-free chess coach where players can announce moves verbally, receive immediate audio evaluation, and practice tactical awareness off-board without needing a sighted partner or visual screen dependence.

What it does

• Hands-Free Speech Recognition: Speak your moves in natural conversational language (e.g. "skoczek f3", "knight f3", "castle", "roszada", "bije e4") in both Polish and English.
• Universal Input Fallback: An accessible typed input sharing the exact same phonetic and semantic parsing pipeline ensures full feature parity across all browsers (including Firefox and offline environments).
• Smart Echo Suppression & Natural Audio Pacing: The voice listener automatically pauses microphone input while the coach speaks to prevent feedback loops. Opponent moves are thoughtfully deferred until the coach finishes verbal delivery to eliminate cognitive audio collisions.
• Tactical Audio Coach: Evaluates positions and vocalizes real-time pedagogical insights, center control, tactical hints, hanging piece warnings, and piece coordination guidance.
• Fast Tactical Responder Bot: An ultra-responsive local Minimax engine delivering tactical replies in under 50 ms to preserve fluid gameplay.
• Grounded Deep AI Analysis: One-click Grandmaster evaluation powered by Google Gemini API (/api/analyze) using grounded prompting (injecting local engine FEN, material balance, and legal moves to substantially reduce hallucination risk).
• Blindfold Practice Mode: Shroud the physical board on demand to train mental visualization, featuring an emergency 3-second peek verification button and a full board audio state reader.
• Audited Tactical Puzzles: Integrated tactical training categories (Knight Forks, Back Rank Mate, Absolute Pins, Skewers, Discovered Attacks, Deflection) with 100% verified FEN geometry and accessible W3C filter groups.

How we built it

• Frontend & Runtime: Next.js 16 (App Router), React 19, TypeScript 5.7, and Tailwind CSS v4.
• Modular Component Architecture: Clean separation of concerns (SiteHeader, ChessBoardView, CoachPanel, TacticsSection, CookieConsent, SiteFooter).
• Speech Stack: Browser-native Web Speech API (SpeechRecognition with Chromium-first optimization; SpeechSynthesis prioritizing local system voices for offline capability).
• Self-Healing Procedural Audio: Procedural move and capture sound effects generated via native Web Audio API oscillators with auto-recovering AudioContext error boundaries.
• Tactical Evaluation Engine: State validation by chess.js and an optimized client-side 2-ply Minimax search with Alpha-Beta pruning, MVV-LVA move ordering, and an O(1) Mating Drive heuristic.
• AI Analysis: Serverless Google Gemini API endpoint (@google/genai) grounded in local engine facts.
• Privacy & Compliance: Fully compliant with GDPR / ePrivacy regulations via a persistent floating consent trigger and Google Consent Mode v2 (cookieless measurement pings when consent is denied).
• Structured Specification: Developed with living planning specifications (prd.md, spec.md, scope.md) and agent-ready llms.txt.

Challenges we ran into

• The 20-Second Engine Freeze ➔ Sub-50ms Turnaround: Early builds executed a 3-ply Minimax search on the main JavaScript thread, causing severe combinatorial explosions (~1.5M nodes) that locked the browser UI for up to 20 seconds. We re-engineered the engine to a calibrated 2-ply search with Alpha-Beta pruning, prioritized MVV-LVA capture sorting, and an O(1) Mating Drive matrix. Turnaround dropped from 20 seconds to under 50 ms, keeping the UI at a fluid 60/120 FPS.
• Web Speech API Fragmentation: SpeechRecognition is cloud-bound in Chromium and absent in Firefox. We solved this by designing a unified semantic parsing pipeline shared identically between voice recognition and an accessible keyboard input bar.
• Audio Race Conditions & Natural Pacing: Handling AudioContext interruptions and preventing speech synthesis from talking over sound effects. Solved with an asynchronous event queue that sequences speech and SFX cleanly.
• W3C ARIA Filter Accessibility: Automated linters flagged tablist/tab structures for lacking dedicated tabpanels. We refactored the puzzle selector into a W3C-compliant role="group" with explicit aria-pressed states.
• Grounded Prompting in Chess AI: Prompting LLMs with raw FENs often causes hallucinated evaluations. By computing evaluation numbers, checks, and legal moves locally and passing them as immutable ground truth, Gemini focuses purely on pedagogical commentary.

Accomplishments that we're proud of

• 100/100 Google Lighthouse Across All Categories: Achieved 100 Performance, 100 Accessibility (axe-core automated suite), 100 Best Practices, and 100 SEO on mobile devices.
• Inclusivity & Accessibility Foundation: Fully keyboard-navigable, high-contrast Staunton SVG vector pieces, zero inline styles, and an established foundation for scheduled manual WCAG 2.1 AA screen-reader audits.
• Sub-50ms Tactical Response: Elimination of UI stutter while preserving tactical sharpness against blunders.
• Privacy by Design: Transparent Google Consent Mode v2 implementation ensuring no tracking cookies or identifiers are written before explicit user consent.
• Fluid Bilingual Support: Instant on-the-fly toggling between Polish and English for UI, speech recognition, and synthesized commentary.

What we learned

• Designing for "voice-first" demands an asynchronous pacing model fundamentally different from traditional touch/click interfaces.
• The vital distinction between automated Lighthouse accessibility scores and true WCAG 2.1 AA conformance requiring manual keyboard and screen-reader walkthroughs.
• Grounded Prompting is essential when integrating generative AI with deterministic logic to mitigate hallucination risks.
• Fine-tuning client-side search heuristics (MVV-LVA + Alpha-Beta) yields instant responsiveness without the download overhead of multi-megabyte WASM engines.

What's next for ChessTactics

• Web Worker Engine Offloading: Moving Minimax calculations to a dedicated Web Worker (chess.worker.ts) to isolate engine computation entirely from UI rendering.
• Blindfold Audio Keyboard Shortcuts: Single-key triggers (Space/R to toggle microphone, S to recite full board coordinates).
• Dynamic Tactical SVG Arrows: Visual on-board directional vectors highlighting threats and coach hints.
• PWA Offline Service Worker: Full client-side caching for offline play in airplane mode.
• PGN / FEN Export: One-tap clipboard export to continue game analysis in Lichess or Chess.com.

Built With

• accessibility
• agent
• ai
• chess
• coding
• gemini-api
• next.js
• react
• tailwind-css
• typescript
• wcag-aa
• web-audio-api
• web-speech-api