-
-
Make a shape: its shadow comes alive on your real wall.
-
Opened on Mia's iPad, in Dad's own voice.
-
Her creature stays on his desk, by the jar of stars.
-
She joins in: the story waits for her hands.
-
Let the light go: the lighthouse wakes. "Goodnight, Mia."
-
Her hand wakes the shark.
-
Carry Mo across to the lighthouse.
-
An octopus: eight arms for eight big hugs.
-
Across the world, her goodnight lands on his wall.
-
Cup your hands and catch the star.
-
Every creature she met waits on the shelf.
-
A ghost hand shows every shape.
-
Catch the star in cupped hands. It glows like a tiny moon.
-
She falls asleep to his goodnight.
-
A goodnight back: a drawing, Mo's colours, and a wish for tomorrow's story.
Inspiration
Every night, parents say goodnight from far away: on military deployment, on a night shift, across an ocean. A video call at bedtime is something, but it is not a story. Everyone knows how to make a shadow rabbit on a wall, so we brought that oldest bedtime game to a child wherever they are.
What it does
A bedtime story for two people in two places, told in the parent's own voice and answered by the child.
- In a Meta Quest, the parent tells the story with their hands. A lamp behind them throws the shadows of their real, tracked hands onto their real wall; each character is a hand shape that comes alive and carries the story on, with Mo, a small moth, riding along. 19 narrated stories in five worlds; 17 hand shapes for 20 creatures. Your room is the stage: the story plays on your own wall, the journal lies on your real desk, and a spatial anchor brings the stage back to the same spot next launch.
- Record and send it with a private family code. No accounts.
- On a phone or tablet, the child watches it in the parent's voice, joins in with their own hands through the camera, and sends a goodnight back: a shadow creature of their own (the classic shapes, or tonight's shark or car), a drawing, Mo's colours and tomorrow's wish ("Dad, tell me a story about Mo at the carnival"). Then the phone becomes a night light: Mo asleep under twinkling stars, fading to dark.
- The next night, the goodnight plays on the parent's wall, and Mo dreams the story the child asked for.
Agentic interaction. Mo is an AI helper, never a replacement for the parent: she plans the child's wish into a story world the app can stage, dreams new stories from a few words, gives a stuck hand one warm tip from what the tracker sees, and suggests stories from each child's taste. She never sees the child, and every line she writes is checked before it reaches the wall.
Social and multiplayer. A parent and a child apart: stories recorded in the parent's voice, Join in, Go live to tell one together and take turns, and goodnights back across time zones (the journal shows the child's bedtime).
Accessibility. One-hand mode, four levels of forgiving shapes, high contrast, Look to press (head gaze) and Voice commands: a whole story can be told without hands, and all of it is on the first screen.
Reason to come back. Every goodnight carries the child's wish, so each night's story ends with the next one already asked for, and the child's creatures and drawing wait on the parent's wall.
First five minutes. No wall of text: a ghost hand shows each shape, help grows at 15, 30 and 45 seconds, and on Quest 3 and 3S the first creature woke 26 to 65 seconds after entering XR.
How we built it
A WebXR app on the Immersive Web SDK (IWSDK) in TypeScript. Plane and mesh detection find the wall and desk; a spatial anchor brings the stage back next launch. Our recognizer matches smoothed joint angles of 25 joints per hand to per-shape templates: instant, offline, and able to explain itself, which is where Mo's tips come from. A Node server on Google Cloud Run shares stories and runs live WebSocket rooms. A child's spoken wish is heard by the browser, or by Google Cloud Speech-to-Text where the browser cannot (Safari on an iPad). Gemini plans, dreams and words Mo's tips; every answer is checked against fixed lists, with simple rules as the fallback. Narration was generated offline with the open Kokoro voice model; the family viewer reads hands with MediaPipe, on the device.
Challenges we ran into
The emulator cannot show you real hands. On a Quest 3 the opening bird took 15 minutes: people make birds with hooked thumbs and relaxed fingers, not the textbook pose. We moved the lamp behind the viewer, as in real shadow theatre, and retuned every shape on headset data; the same bird then woke in 6 seconds.
Accomplishments that we're proud of
- Proven on Quest 3 and Quest 3S in 15 diagnostic sessions: 72 Hz held, only 1 of 1,754 five-second windows under 60 fps (entering XR), and 142 creatures woken, every one the shape the story asked for. Every session is in DEVICE-RESULTS.md; the demo in under 3 minutes: here; full sessions, uncut: Quest 3 and Quest 3S.
- A story that never fails: a ghost hand shows each shape, and help grows at 15, 30 and 45 seconds.
- The family side tested on an iPad, a computer and an iPhone.
What we learned
Real rooms and real hands are the spec. Privacy-safe diagnostics (?diag: numbers only, never audio, video or names) turned every headset session into a list of fixes we could verify the next day.
What's next for Bedtime Across the World
A Horizon Store release as an installable web app, more stories and languages, a library of stories recorded by grandparents to play any night, and testing with families who live apart.
Built With
- docker
- gemini
- google-cloud
- google-cloud-run
- google-cloud-speech-to-text
- google-cloud-text-to-speech
- hand-tracking
- iwsdk
- javascript
- kokoro
- mediapipe
- meta-quest
- mixed-reality
- node.js
- playwright
- three.js
- typescript
- uikit
- vite
- web-audio
- websockets
- webxr