Lantern Hand is a short story you play seated, in your own room, with one bare hand: hold out your palm and it becomes a paper lantern, and a small paper traveller called Wick crosses your real furniture toward its light.

Entertainment. Immersive storytelling where the stage is the room you're sitting in. Wick wakes on your table, walks the back of your couch, sails a paper boat across your floor and watches the sun come up on your windowsill, with narration and sound coming from each place. He stays where you left him: come back and chapter two starts on that sill, where you hang paper stars for him to climb to the moon.

Inspiration. Quest owners keep telling Meta that their hands get in the way of media, and Meta's own answer on its forum was to turn hand tracking off. We wanted the opposite: a lean-back story where the hand is the whole input, and the room the headset already understands is the set.

How it works on Meta's platform

  • Scene understanding (WebXR plane detection, Immersive Web SDK): every beat lands on a labelled surface from your room scan, in front of you and never hidden behind furniture, with a fallback for every missing label. app/src/room/stage.ts
  • Hand tracking: an open palm lights the lantern and pointing casts the light, with aim assist because hand tracking is noisy. No pinch anywhere, so the system menu never opens by accident. app/src/story/hands.ts
  • Passthrough: night falls over your real room and lifts at dawn.
  • Persistent anchors: his resting place is pinned to your real room, so next visit he's on the same spot.
  • Gaze: with no hands in view, looking at the glow lights it, so it can be played without hands.
  • Meta VR Glasses: captions and beats stay inside the Glasses' design area; both chapters pass in the emulator's Glasses profile.
  • No room scan, or half a scan? Paper stand-in pieces fill the gaps, so it always starts.

Check it yourself

  • Play it: https://lantern-hand.vercel.app in the Quest Browser.
  • Our own break test of the staging (we tried to make it fail), rerun with one command: npx -y tsx evidence/eval_staging.ts.
  • Hands-only, knock and gaze-only walkthroughs of both chapters in Meta's emulator: cd app && python3 tests/walkthrough.py --knock and --gaze.

Evidence, stated plainly: no real users yet; it was built for this hackathon. Instead: our own break test of the room staging, including room scans it was never tuned on. It reruns from the repo with one command.

Claim: Lantern Hand stages a full chapter in 401 of 401 test rooms: 8 emulator room scans with the viewer at fixed and random seats, heights and facings, stripped-down and partial copies of every scan, and the stand-in room. Evidence type: Our own break test (we tried to make it fail), not real users. Data: the eight room scans in Meta's Immersive Web Emulator plus 16 fixed and 240 random seats, stripped-down and partial copies, and the stand-in room: 401 cases (6 of the 8 scans were never used for tuning; 22 random seats facing a wall under 0.9 m away were out of scope). Baseline: our own naive fixed-offset staging, 0 of 401. Result: 401 of 401 (95% interval 99.1% to 100%). Rerun: npx -y tsx evidence/eval_staging.ts. Limits: emulated scans, not real headsets; the degraded copies come from the same rooms; no frame-rate or comfort measure on a device.

What doesn't work yet

  • Tested in Meta's emulator only, not on a physical headset, so frame rate and real-room behaviour are unverified. What we did measure, in the emulator: 28 to 38 draw calls and about 20,500 triangles per frame across both chapters.
  • Two chapters so far; chapter three is not written.
  • The emulator rooms give labelled flat surfaces but no labelled furniture meshes, so staging uses flat surfaces only.

What's next. A chapter each night, a test on a real Quest, and a launch alongside Meta VR Glasses in spring 2027.

Art: paper cut-outs made with an AI image model. Narration, sound effects and music made with an AI voice and sound model.

Built With

Share this project:

Updates

Submission history