Inspiration
We wanted a tabletop war game that feels alive. Card games are quick to learn, but the battles they describe stay on paper. We put the battle on your own table, with fighters that move like real people instead of canned animations.
What it does
King Of The Cards is a mixed-reality card battle for Meta Quest 3 and 3S. You place a battle table on your real desk or floor with a pinch. Your hand of cards fans out over your left wrist. Take a card with your right hand, carry it over the battlefield and let go on a gold ring to play it: units march for the hill in the middle, and buffs like Explosion and Meteor change the fight. The army that holds the hill wins.
You play it seated, within arm's reach, with hands or controllers.
Agentic AI runs the motion and animation
Motion is the heart of the game, and AI handles it from start to finish.
- In the game, every fighter is an autonomous agent. The battle simulation gives each one its goal (march to the hill, choose a target, strike, block, dodge, get up after a blast), and a learned motion model (MotionBricks, run through ONNX Runtime in our native C++ plugin) generates its whole-body movement frame by frame. There are no keyframed clips: every step, swing, fall and recovery is generated live, so no two fights look the same.
- In production, a team of AI agents built and tuned the motion. We used agentic AI (Claude Code) as our animation and tech-art crew: one agent per feature, a supervising agent that reviewed each result, and us approving every model and animation before it shipped. The agents integrated the motion model into Unity, fitted 14 weapon models to the fighters' hands (including the bow's string and draw), tuned the ragdoll throws and get-ups after explosions, and built a sandbox of staged battle scenes so every motion change was checked with rendered pictures, not just numbers.
How we built it
- Unity 6 with OpenXR, passthrough and hand tracking on Quest 3 / 3S.
- A C++ battle simulation with a learned motion model (MotionBricks + ONNX Runtime), so every fighter's movement is generated, not keyframed. Unity hosts it through a native plugin.
- Agentic AI (Claude Code) as the development crew for motion, animation, weapons and testing, with a human approving every result.
- Physical cards: a wrist-mounted fan, pinch-to-grab, and drop targets on the table.
- Online play through a small UDP relay server; a PC host runs the simulation and the Quest draws it.
- All sounds are synthesised in code; the card art, icons and UI are our own.
Challenges we ran into
- Running a learned motion model in real time on a standalone headset: the Quest run the AI model in the chip and predict the animations.
- Picking the right card from a tight fan with a pinch, without taking its neighbour.
- Keeping the battle readable at table scale: we halved the field and made units life-size so feet stop sliding.
Accomplishments that we're proud of
- Fighters with generated, natural motion on a real table in mixed reality.
- A full card-to-battle loop you can play with your hands.
What we learned
- Hand interaction changes card design: cards must be big enough to pinch one at a time, and drop targets must be clear from a seated view.
- A standalone headset sets a hard budget, so we split the work: the simulation runs on a PC host and the Quest draws the battle.
What's next for King Of The Cards
- A standalone Play vs AI mode on Quest.
- More card art and unit types.
- Matchmaking without a PC host.
Built With
- agentic-ai
- c#
- c++
- claude-code
- hand-tracking
- machine-learning
- meta-quest
- mixed-reality
- motionbricks
- onnx-runtime
- openxr
- passthrough
- unity