MagicBox

Inspiration

Say an incantation into your mic, then throw the spell at your opponent with a push toward the camera. Magic Box is a two-player duel played with voice and body movement.

What it does

Two players stand side by side at one monitor. Each half of the screen shows a live close-up of the opponent. At startup, the game checks the camera, both mics, network and API key. In the lobby, players say their names and choose Fire Mage, Lightning Mage or Holy Knight by voice, then raise their hands to begin. Muse Spark writes three incantations per player, themed to their name and class, and the loading screen shows how to cast each one.

The fight lasts ~90 seconds. Saying an incantation into a clip-on mic charges a spell; releasing it takes a fast push, a two-hand push, an arm overhead or hands together at the chest. The defender can see the projectile coming and step aside or duck. After the fight, the screen shows the winner, both players' stats and a Duel Chronicle written by Muse Spark from the match log.

The camera measures the wrist's peak speed toward it relative to the neck, so walking forward alone doesn't count as a punch. A harder punch does more damage. Distance matters for attacker and defender alike: the multiplier runs from ×1.4 at 1.2 m to ×0.7 at 2.8 m. A dodge takes a 0.3 m sidestep or a 0.25 m duck in the last 0.6 s of a projectile's flight.

Each class has three spells. Six have effects beyond damage: a shield against the next hit, a three-bolt Chain that requires three dodges, slower mana regeneration or a shorter charge window for the target, a heal, and a damage buff. F1 opens a debug overlay with both players on a top-down map in meters and a graph of each wrist's speed against the push threshold.

How we built it

A Stereolabs ZED X connects over GMSL2 through a ZED Link capture card to a Jetson Orin Nano. The Jetson drives the monitor and runs the game; each player has a clip-on mic. There is no laptop in the demo loop.

Three supervised processes start with spawn, and a crashed child restarts. Perception alone accesses the ZED SDK. It tracks two bodies, assigns player slots, detects pushes and poses, and puts camera frames in a shared-memory double buffer. Voice keeps a Muse Voice Transcribe WebSocket open for each mic; it sends that player's incantation words as keywords and fuzzy-matches partial transcripts against their spells. The pygame-ce game loop fuses voice and body input, runs the rules and draws the screen without blocking. Muse Spark runs in background threads with an 8 s deadline and fallback text.

Typed dataclasses carry messages between processes and round-trip to JSON lines. We could build on laptops using --fake, with keyboard players and a synthetic camera, then use --record and --replay for real sessions. The game has a HUD for each player, five scenes, hit and K.O. effects, projectiles with glows and trails, and sounds synthesized with NumPy at startup. Titles use Cinzel Decorative (SIL Open Font License).

We run 960 automated tests headless in GitHub Actions alongside Ruff lint and format checks. Thirty end-to-end tests play whole fake matches on a fake clock. A soak script plays random matches and watches frame rate and memory use.

Challenges we ran into

Muse Spark initially missed our 8 s deadline when generating six incantations with its default reasoning setting. With reasoning_effort: minimal, it took 3.1-3.8 s and all six incantations passed validation in three runs on laptop Wi-Fi.

Muse's cumulative partial transcripts can match a spell after only the first few words. That meant the full-phrase "perfect incantation" bonus rarely fired. We now wait up to 0.5 s on a sub-perfect match to give the rest of the phrase time to arrive. We also found that one keyword longer than 30 characters makes Muse reject the entire session handshake, so we send single words.

Speech and gestures arrive on different clocks. In our laptop tests, the charge arrived about 0.4 s after speech ended. If someone punched on the last word, they got a basic attack instead of their spell. While that player's mic is on, we now hold an uncharged push for up to 0.6 s. For punches, reading speed as soon as it crossed the threshold made nearly every hit look alike. We wait up to 0.1 s for the wrist's peak speed instead.

The GMSL2 camera connection isn't hot-pluggable. Re-plugging it while the Jetson runs requires a daemon restart, so we taped down the cable.

Accomplishments that we're proud of

We have the full sequence from boot and voice-controlled lobby through Spark-written incantations, a voice-and-body fight and the match chronicle. Both Meta APIs were verified live on a laptop. Using synthesized speech, Muse matched the correct spell in all 17 test sessions and made no wrong matches. Spell keys and built-in text for incantations and the chronicle let the game run when venue Wi-Fi fails. A laptop test also measured 0.27 s to quit with a Spark request still in flight.

What we learned

We want to test with more players and tune the push threshold, dodge distances and charge window. We'd also like more classes, spells and gestures beyond pushes, a single enclosure for the camera and screen, and a spectator view on a second display.

What's next for temp

We want to test with more players and tune the push threshold, dodge distances and charge window. We'd also like more classes, spells and gestures beyond pushes, a single enclosure for the camera and screen, and a spectator view on a second display.