Inspiration

Rock, Paper, Scissors is one of the few games that everyone already knows. No tutorial. No controller. No learning curve. You just throw a gesture with your hand and start playing.

That made me wonder: what if those gestures weren't just symbols anymore? What if Rock, Paper and Scissors became actual fighters battling it out on your real table through mixed reality?

HotHandZ started as an experiment to answer that question.

What It Does

HotHandZ is a mixed reality tabletop fighting game for Meta Quest that uses hand tracking and passthrough. After calibrating your table, a miniature arena appears right in front of you.

Each round, players throw a Rock, Paper or Scissors gesture to summon a unique fighter:

Rock becomes the Titan, a heavy hitter that deals 30 damage. Paper becomes the Mystic, protected by a shield and dealing 20 damage. Scissors becomes the Assassin, a fast attacker dealing 25 damage.

A dramatic 5-4-3-2-1 countdown precedes every clash. The losing fighter doesn't just lose, they're engulfed in flames.

You can challenge AI opponents or battle players online in real time. Every player starts with 100 HP, with ranks, hot-streak achievements and a global leaderboard tracking the competition.

How We Built It

HotHandZ was built using Immersive Web SDK 1.1.0-rc.2 on Three.js r185, alongside Vite and TypeScript in an entity-component-system architecture. The game's interface is built with UIKitML.

All fighters and arena pieces are generated procedurally rather than imported as traditional 3D models. This allows the entire experience to adjust dynamically to tables of different shapes and sizes.

For multiplayer, I built an authoritative Node.js WebSocket server that controls timing, player choices and damage calculations. By keeping the game state on the server, matches stay synchronised and resistant to cheating.

The flame effect is powered by a custom GLSL shader using animated noise, displaced geometry and a stylised colour ramp to create fire that feels alive.

Challenges We Ran Into

The fire effect took far longer than expected. My first versions looked nice, but they felt decorative rather than dangerous. What finally worked was a two-stage effect: an explosive burst of particles followed by flames that ignite in random places and sizes.

Some of the hardest bugs were the quiet ones.

One of our biggest lessons was learning that the editor isn't the game. The scene editor can show how things look, but only the runtime reveals how they actually behave.

Accomplishments I'm Proud Of

The biggest win is that the game is live and playable right now.

Players can open a URL on a Meta Quest headset and immediately jump into a hand-tracked mixed reality multiplayer game. No installation required.

What I Learned

Hand tracking taught me that accuracy isn't everything.

A gesture recogniser that demands a perfect pose feels broken, even when it's technically correct. The experience became much smoother when i started accepting natural variations. For example, a Rock gesture is still recognised whether the thumb is tucked in or not.

What's Next for HotHandZ

My next goal is local pass-and-play, allowing two players in the same room to share a single tabletop arena.

I am also exploring spectator mode so players can watch live matches unfold in mixed reality.

And, of course, i'm preparing for a full Meta Horizon Store release. The packaging and signing work is already complete, so i'm getting ready for the final step.

Built With

Share this project:

Updates

Submission history