Inspiration

In Mexico, on DĂ­a de Muertos, we set an ofrenda for our dead: candles, cempasĂșchil flowers, pan de muerto, sugar skulls, papel picado. Calacas, the cheerful skeletons of that day, say that death can be laughed with. Mexico also has a long tradition of string puppets. I wanted to put both in your hands.

Hands in VR mostly grab, poke and pinch. A marionette asks for something else: open hands held in the air, every finger with a job. I looked for a marionette game on the store and could not find one.

What it is

You sit in front of a tiny theater on an ofrenda. A pair of pale hands holds the puppet; hold yours over the stage and the strings pass to you, cut to the height that is comfortable for you. Your wrists carry the body, your index fingers lift the legs, your middle fingers lift the arms, and tipping your hands forward makes the calaca bow. The audience, eight sugar skulls, asks for moves on a sign: wave, walk to the star, dance to the beat, kick, sit, fly, play dead... eleven moves in five acts. A band plays all along; a step that lands on the beat knocks like a wood block. When you do what was asked, the skulls hop, chatter their jaws and throw marigolds on the stage. Lower your hands and the pale hands take the puppet back: the show waits for you. One hand is enough: its two fingers move both sides.

How I built it

With the Immersive Web SDK (WebXR), in TypeScript. There is no physics engine: the puppet is a small simulation of points, rods and strings that only pull, written for this game, so a string goes slack exactly when a real one would. The bow needed a real puppeteer's trick: a string tied to the tailbone, which takes the weight when the hands tip. Everything you see is made in code, with no downloaded models: the skeleton and its charro suit, the ofrenda, textures painted pixel by pixel. Everything you hear is made on the spot too, with Web Audio: an original tune for marimba, bass and strummed chords, and the game keeps time from the audio clock.

I built it with an AI coding agent (Claude Code) that wrote the code under my direction; I chose what the game is and judged every step from recordings. It was built with the SDK's headset emulator, with scripts that move the emulated hands, and then played on a Meta Quest 3 with real hands: the strings pass to your fingers and it holds 90 frames a second.

What comes next

More puppets with their own bodies and tricks (La Catrina first), more stages, harder acts that chain moves, and a duet: two players, a puppet each, on the same stage.

Built With

Share this project:

Updates

Submission history