Inspiration

Everyone has drawn a stickman fight in the margin of a notebook. We wanted to see those fights for real: your own drawing, alive, brawling on the table in front of you.

What it does

Drawn to Fight is a mixed reality game for Meta Quest, played entirely with your hands.

  1. Draw a fighter with a marker on real paper. Add a sword, a gun or horns if you like.
  2. Show it to the headset and pinch. The drawing gets traced with light and comes alive.
  3. Pull your fighter up out of the paper and drop it on your real table.
  4. Watch them brawl. Fighters punch, kick, dodge and finish each other with motion-captured moves. A finishing blow tears limbs off in a splash of ink.
  5. Bet. A coin purse sits on your left hand. Toss coins onto the fighter you believe in. Every kill doubles your stake.

Pick a fighter up with a slow pinch, throw it back into the fight, or swat it away with an open hand. No controllers, no menus.

How we built it

  • Unity 6 + Meta XR SDK: Passthrough Camera API to photograph the drawing, Interaction SDK for grabbing and throwing, MR Utility Kit for the real table.
  • Claude reads the drawing. The photo goes to a small server where Claude looks at it and calls a create_fighter tool with the skeleton, proportions, props and personality. The server validates the spec with zod and hands errors back to Claude to fix. AI does the rigging, so there is no pose detection to train.
  • Physics ragdolls driven by motion capture: about 170 real punches cut from shadow boxing clips, plus dodges, finishers and kung fu kicks, retargeted onto a 9-bone stickman.
  • Cloud: a Cloudflare Worker in front of a Docker server; the API key never touches the headset.
  • Headless test harness: Unity plays fights in batch mode and logs every fall and hit, so we tune by data, not by guessing.

Challenges we ran into

  • Fighters falling through the table on spawn. A headless audit took it from 11 to 17 falls per 180 s down to zero.
  • Making a fast hand hit but a slow pinch grab, so the same hand can both fight and carry.
  • Keeping fights readable on a desk-sized arena: dodges capped to the table, no falling off the edge.

What we learned

Scoping to one mechanic matters. We started with a different idea (AI dares in your room) and pivoted to the moment that made people laugh in the headset: their own drawing coming alive.

What's next

Multiplayer betting over one table, tournaments of saved fighters, and more drawn weapons.

Motion Data by Bones Studio.

Built With

  • anthropic
  • c#
  • claude
  • cloudflare-workers
  • docker
  • interaction-sdk
  • mcp
  • meta-quest
  • meta-xr-sdk
  • motion-capture
  • mr-utility-kit
  • node.js
  • openxr
  • passthrough-camera-api
  • typescript
  • unity
  • zod
Share this project:

Updates

Submission history