Inspiration

Hand tracking on Quest is good enough to fight with, but most hand-tracked games still treat your hands as pointers. We wanted the opposite: a game where your hands are the controller, the weapon and the spellbook, and where you never reach for a button.

We also wanted it to work sitting down. A desk gives your hands something solid to rest on, a reference for calibration, and a comfortable way to play for people who can't or don't want to stand and swing for 10 minutes.

The theme came from tarot. Every boss is a Major Arcana card, and you're card 0, The Fool: the one card that can't be rewritten. That gave us a roguelite loop with a reason behind it. The tower changes every run because something up there rewrites it.

What it does

You are locked in a cell at the bottom of a tower. To get out:

  • Dig. Punch through rock, clap both palms to send a shockwave that brings the loose rock down, and wipe the rubble away.
  • Cast. Six spells, six hand signs: a palm push, a double punch, a double palm push, a finger gun, a wipe and a kung fu salute clap. Each one breaks a different material. Hold a sign still to charge it: every spell has a Tier 2, and Ember climbs to four tiers (Ember, Kilnburst, Inferno Rune, Meteor), each breaking more of the wall.
  • Break wards. Magic brick walls made of ice, stone, sandstone, iron and root. Find the keystone, hit it with the right spell and the wall comes down.
  • Fight the Warden. Brother Grout is a life-size stonemason monk. Lean and duck to dodge, raise your fists to guard, punch just as he swings to parry, punch him when he's open. Each of his earth arts has a hand answer: punch through his Mortar Wall, punch his lobbed rocks out of the air (Hod Load), clap three times to crack the mortar cuffs he sets on your hands (Set in Mortar), brace both palms flat on the desk against his Foundation Stomp, and punch your way out when he Entombs you. Stars you earn power a Thunderclap: clap both palms together for a shockwave. In phase 2, rune slabs orbit him and you punch them out of the air.
  • Bind him. Beat a Warden and he's sealed into his tarot card. Your cards stay with you; the tower doesn't.

Three floors and three Wardens (Brother Grout, Madame Frostwick, the Master) make up Act I. Each run builds a fresh tower with shops, shrines, a forge, an upgrade tree, Glyph and Curse cards.

No controllers anywhere, including the menus: you touch buttons with a fingertip.

A Practice & Tests mode has a sign drill, a 10-minute "desk test" that counts accidental casts while you type and drink, and a dodge drill.

How we built it

  • Unity 6 (URP) with OpenXR, XR Hands and Meta OpenXR for hand tracking and passthrough on Quest 3.
  • A data pack generated offline in Python. The sign classifier, the duel rules, the boss arm IK and choreography, the towers, the pixel-art UI and the icons are all generated by Python scripts. The C# ports have to reproduce the Python fixtures exactly: 48 duel replays, 200 IK cases, 54 sign gestures.
  • The sign classifier reads finger curl, palm direction and hand speed from the XR Hands joints. A pairing layer combines both hands within 0.2 s, so a single fist stays a punch and two fists become Stonefist. The clap is read every frame from palm distance and closing speed. We tuned it against 60 synthetic players and then against logged real casts.
  • Logging built into the game. Every session writes frame times, every sign read with the hand frames it came from, every fast move that didn't cast, dodges, tracking losses and a summary. Logged casts replay through the classifier as EditMode tests, so a rule change shows up as a diff.
  • A forgiving desk calibration. Flat hands are the hardest pose for hand tracking, so the start-up check adds up good frames instead of needing one unbroken second, ignores one-frame tracking jumps, and shows a ring under each hand that fills as it holds.
  • Spell effects are built in code: missiles, ribbons, arcs and impact decals, with a different shape and sound for each element.
  • Audio: sound effects layered from Kenney's CC0 packs plus synthesised fire, water, wind and lightning. Music by Kevin MacLeod (incompetech.com, CC BY 4.0). Character voices with ElevenLabs.
  • Video: the trailer is cut from real Quest 3 recordings and in-engine captures by a Python + FFmpeg script, with pixel-art story slides drawn in code. Brother Grout's whole fight in the video is a real Quest 3 playtest, hands only.

By the numbers

From one 14.7-minute session on Quest 3 (Nov 3, 2026):

Digging Boss fights
Average frame rate 71.8 fps 71.9 fps
p95 frame time 13.9 ms 13.9 ms
Frames over 16.7 ms 0.08% 0.05%
Minutes logged 7.7 5.6

Across all logged Quest sessions, boss fights and menus held p95 at 13.9 ms. One 22-minute dig session on an older build hit 27.8 ms p95; every dig session since is at 13.9 ms.

Casts per spell in that session: Spark 374, Ember 169, Gust 99, Stonefist 15, Tidepush 5 (Steam claps are detected separately and aren't in this log). The headset saw 9,355 fast hand movements and turned 888 of them into sign reads: punches and reaches don't cast.

Sign classifier on synthetic players: 89–98% of each sign recognised, at most 0.25% read as the wrong spell, and every idle-hand distractor (typing, drinking, reaching, waving) fires at most 1.6% of the time.

Tests: 109 / 109 EditMode tests pass.

Playtests

20 logged Quest sessions, 136 minutes in all (Nov 2–5). The logs (every sign read, every dodge, every fight's result) drove most of the tuning:

Fights on the headset Result
Brother Grout (Floor 1) 12 11 losses, then a win in 2 min 28 s
The Master (Floor 3) 1 won first time, 3 min 5 s
Madame Frostwick (Floor 2) 0 not played on a headset yet
  • Grout's first fights ended in under 40 seconds (six of them). We made every Warden hit softer (60%), doubled your vitality and mana, and let Grout go down faster.
  • Punches didn't reach him. Seated, your fist gets to about 0.6 m and he stood 1.2 m away. He now stands in arm's reach, and a real punch lands on his head, his body or his shield stones.
  • Dodges were hard to time, so every Warden can now also be parried: punch at him in the last part of his wind-up. Timing is still the hardest part: in the session with both wins, 17 attacks were answered and 23 missed.
  • Phase 2's orbiting stones were out of reach. The orbit now passes 0.4–0.6 m from your eyes, and each stone you punch hurts him.
  • Getting up after a knockdown took six fist pumps; now one raised fist.

Floors 2 and 3 are tuned in the editor only: the Master has one headset fight, Madame Frostwick none, and the Floor 2 dig hasn't been played on a headset. That's the next playtest.

Challenges we ran into

  • Telling a punch from a spell. A hard punch with a loose fist used to read as a fire sign. We moved Ember to an open palm, made Stonefist need both fists, and added the 0.2 s pairing window.
  • Reach. Seated, you can't step toward a boss. Grout was too far to hit, so the arena now pulls him in and his phase 2 stones orbit within punching distance.
  • Spells that all looked like balls. We rebuilt every effect so each element has its own shape, cast burst, impact and sound.
  • Session length. Our first full run took longer than 10 minutes. We cut the rock between rooms on Floor 1 so a newcomer reaches Grout in about 8 minutes, without shortening the fight.
  • Getting past the first screen. Calibration with both hands flat on the desk took up to half a minute when tracking flickered. We rewrote it to forgive short dropouts and show the player what it's waiting for.

Accomplishments that we're proud of

  • No controllers anywhere in the game, including menus.
  • A steady 72 fps on Quest 3 with life-size bosses, physics rubble and particle spells.
  • A sign set that people can learn in a minute and that doesn't fire while you're typing or drinking coffee.
  • Boss attacks that each ask for a different thing from your hands: punch, clap, brace, parry.
  • Logs that let us answer "why didn't that cast?" with the actual hand frames.

What we learned

Hands-only design is mostly about what shouldn't happen. The hard work was rejecting the 90% of hand movements that aren't spells, and making the ones that are feel instant.

What's next for Wardbreaker

  • Headset playtests of Floor 2 (Madame Frostwick) and Floor 3, then tune them the way Grout was tuned; record the remaining voice lines.
  • Act II: the Iron Citadel and the Emperor, whose Decrees forbid some of your signs.
  • Daily tower leaderboards and shareable runs.
  • More signs and two-hand combos, and room-scale MRUK desk detection.

Target launch: early access on the Meta Horizon Store in Q1 2027.

Built With

Share this project:

Updates

Submission history