Inspiration
Everyone who grew up on wizard stories has waved a stick and shouted a made-up spell. We wanted to make that actually work. Most “magic” games reduce spellcasting to pressing a button, but the fun of magic is the performance: the motion, the words, the timing. So we built Wander, a world with its own original spells and lore, where you physically draw spells in the air and speak them aloud. We also wanted the opponents to feel alive, so our enemies learn how you fight instead of following a fixed script.
What it does
Wander is a spellcasting duel you play with your body and voice.
Cast with gestures: draw shapes in the air with your fingertip, tracked by a webcam, or with our custom hardware wand. A circle raises Aegis Ward, a lightning zig-zag hurls Storm Lance, a triangle freezes your foe with Rime Spire. There are 7 spells in all.
Speak the incantation: say “Fulmina!” while drawing and the spell becomes Empowered (1.5× power). Voice alone casts a weaker version.
Fuse spells: chain two spells within 4 seconds to trigger one of 6 fusions. Rime Spire followed by Storm Lance becomes Shatterstorm, which deals huge damage to a frozen enemy.
Duel an adaptive AI: Morvane, the Hollow Archon models your play style: favorite spells, the order you cast them in, and how often you block or interrupt. It counters accordingly, and its “Mind” panel shows its reasoning live (“You block 50% of my hexes. Wardbreaker.”). It even remembers you between duels.
Arena multiplayer: a free-for-all for up to 4 wizards over local Wi-Fi. Empty seats fill with AI bots, each powered by a different model (Gemini, Claude, GPT, Grok or Meta Llama). Every bot plans its own strategy and trash-talks in character.
The wand: an Arduino Nano with an LED tip the webcam tracks and an OLED screen that shows your current spell, your health, and reactions like “Clean form!” or “OUCH!”.
Spell Forge: invent your own spell by drawing a new shape a few times, then give it a name, an incantation and an effect.
How we built it
Game: Python and pygame, rendering at native resolution with an additive-glow particle system, procedural lightning, and every sound effect synthesized in numpy (no asset files).
Hand tracking: MediaPipe HandLandmarker on a background thread. We classify pointing and pinching from 3D joint angles, debounce poses, smooth the fingertip with a One Euro filter, and coast through dropped frames so a single missed frame doesn’t break a stroke.
Gesture recognition: a $1-Unistroke-style template matcher plus geometric heuristics (straightness, turning angle, radial variance) for shapes templates struggle with, like spirals and straight lines.
Voice: continuous speech recognition with fuzzy matching, because speech-to-text doesn’t know our invented words (“Fulmina” often comes back as “full mina”).
Wand hardware: an Arduino Nano, a button, a 128×128 SH1107 OLED and an LED tip. OpenCV finds the brightest small dot in each webcam frame and learns to ignore lamps and windows. A 4-corner calibration maps the camera view to the screen. A serial protocol sends the button state to the PC and spell info back to the OLED.
Multiplayer: a server-authoritative rules engine over WebSockets. Each AI provider gets its own strategist thread that returns structured JSON plans, while each bot’s fast reactions (shielding, cleansing) stay local
Challenges we ran into
Invented words vs. speech recognition: we needed alias lists and fuzzy matching. Network speech also lags 1–2 seconds behind a gesture, so we added a window that matches the spoken word to a gesture whichever comes first.
Making MediaPipe reliable: strokes kept breaking on single missed frames, and poses flickered. Dropout coasting, majority voting and 3D-angle pose detection fixed it. A MediaPipe update also removed the API we started with, so we moved to the newer Tasks API.
AI latency in a real-time game: full-size models took 7–40 seconds per reply, far too slow for a live fight.
Lightweight “flash-lite” models answered in about 1 second, and splitting slow strategy (the AI) from fast reflexes (local code) kept bots responsive.
Sharp graphics on high-DPI screens: Windows display scaling blurred the game, and rendering at native resolution first dropped us to 15 fps. Caching and redrawing the HUD less often got us back to roughly 55–75 fps.
Hardware surprises: Many laptop webcams filter out IR light.
The webcam driver kept our dark “wand mode” exposure even after the game closed.
A calibration bug came from mixing two different clocks.
The Nano’s 2 KB of RAM forced page-by-page OLED drawing.
Accomplishments that we're proud of
Casting genuinely feels like magic: draw a triangle, shout “Glacia!”, and watch the enemy freeze. An enemy that visibly learns you, and tells you what it has learned.
Five different AI models fighting in one arena, each with its own personality and strategy.
A working physical wand that ties hardware, computer vision and the game together.
Everything degrades gracefully: no webcam means mouse input, no microphone means keys, no API keys means built-in bot tactics.
What we learned
Good input design matters as much as graphics. Smoothing, debouncing and forgiving timing windows are what make gesture controls feel natural.
LLMs work best in real-time games as strategists, not as the reflexes.
Hardware is humbling. Exposure settings, IR filters and limited RAM all mattered more than we expected.
Test everything you can without the real device: we simulated a webcam and a whole Arduino to debug the wand before touching hardware.
What's next for Wander
Wireless wand: a battery and Bluetooth instead of USB, plus a motion sensor to detect flicks and twists.
Online play: a hosted Arena server so friends can duel from anywhere, not just on the same Wi-Fi.
Bring it to the browser: port the full game to the web version so anyone can play from a link.
More magic: new spells and fusions, a story campaign, and sharing custom Forge spells with friends.
Log in or sign up for Devpost to join the conversation.