MindGym: The Cognitive Newspaper

Inspiration

In an era dominated by artificial intelligence, the human capacity for critical thinking faces a paradox: as our tools get smarter, we risk becoming more passive. I noticed a decline in "mental friction"—the productive struggle required to learn logic, spatial awareness, and deduction.

I asked myself: What if I used Superintelligence not to do the work for us, but to act as the ultimate personal trainer for our brains?

MindGym is a resistance against cognitive entropy. It is a daily "Gazette" of protocols designed to stimulate dormant synapses, wrapped in a tactile, 1920s-meets-Cyberpunk aesthetic.

What it does

MindGym is a suite of multimodal cognitive games ("Protocols") where Gemini 3 acts as the game engine, the opponent, and the coach.

1. The Intel Briefing (Search Grounding)

Gemini 3 scans real-world news using Google Search Grounding and correlates disparate events into a daily logic puzzle. Users must find the hidden link between global headlines.

2. Live Mentor "The Editor" (Gemini Live API)

A real-time, low-latency voice conversation with an AI coach. We used the Gemini Live API (gemini-2.5-flash-native-audio-preview) over WebSockets to create a persona that listens to you while you solve puzzles. It doesn't give you the answer; it guides your reasoning process using Socratic questioning.

3. Field Ops (Multimodal Vision)

A scavenger hunt where users find physical objects (e.g., "a vessel for heated liquid"). The app uses the camera and Gemini 3 Vision to analyze the video feed in real-time, verifying if the object matches the mission criteria.

4. Neural Sudoku & Deduction Grids (Reasoning Core)

Classic logic games enhanced by Gemini 3's Thinking Mode. Instead of hard-coded hints, we allocate a thinkingBudget of 8,000 tokens. The model analyzes the board state, simulates potential moves, and provides a "Deep Reasoning" hint that explains why a move is valid, rather than just telling you what to play.

5. Neural Sculptures (Imagen)

A reward system where your daily aptitude score is converted into a unique 3D-style trophy generated by Gemini 3 Image models. Users can then use the "Refine Artifact" feature to edit their trophy using natural language prompts.

How we built it

I built MindGym using React, Vite, Tailwind, and Three.js (React Three Fiber) for the immersive 3D "Gazette" interface.

The core intelligence is powered by the Google GenAI SDK:

Deep Reasoning: I used gemini-3-flash-preview with thinkingConfig to power the logic engines for the Deduction Grid and Sudoku.
Audio Streaming: I implemented a custom AudioWorklet to handle raw PCM 16kHz streaming directly to the Gemini Live API, allowing for sub-200ms latency voice interaction.
Prompt Engineering: I utilized structured JSON schemas to force the models to generate valid game data (Sudoku boards, crossword clues, and logic puzzles) deterministically.
Visuals: I wrote custom WebGL shaders to create the "Fluid Ink" background effect that reacts to mouse movement, creating a seamless blend between the retro newspaper aesthetic and modern tech.

Challenges we ran into

Audio Encoding: The Live API requires specific raw PCM audio formats. Implementing the binary encoding and decoding manually in the browser without external heavy libraries was a significant technical hurdle.
Thinking vs. Latency: Balancing the thinkingBudget was tricky. Too high, and the hint took too long; too low, and the reasoning wasn't deep enough. I found the sweet spot at 8k tokens for real-time game assistance.
3D Performance: Integrating heavy Three.js shaders alongside heavy AI API calls required careful memory management and lazy loading of game components.

Accomplishments that we're proud of

The "Ink" Aesthetic: The UI feels like a living, breathing newspaper.
True Multimodality: The app genuinely uses text, audio, image, and video inputs/outputs seamlessly in a single session.
The "Deep Hint" System: Seeing the model "think" through a Sudoku puzzle and give a human-like nudge ("Look at the center column, you're missing a 5") felt like magic.

What's next for MindGym

Multiplayer Lobbies: Solving the "Daily Gazette" with a friend in real-time.
Long-term Memory: The Live Mentor remembering your specific weaknesses (e.g., "You always miss the knight moves in Chess") across sessions.
AR Integration: Expanding "Field Ops" to overlay puzzle elements onto the real world.