Inspiration

I wanted something for the end of a long day: no score, nothing to fail, just a small, kind choir that sings to you in your own space. Mixed reality makes that personal. The robots float in your living room or beside your seat, at arm's reach, and each voice comes from its own little head.

What it does

Soothing Bots is a hands-only mixed-reality choir of 16 robot heads, a neighbourhood of characters: a family, their grandparents, aunties and neighbours. Put on the headset and the song starts.

  • Immersive storytelling: twelve story-song episodes, such as Free Refills, The Big Box, Magic Beans and Murphy's Lucky Day. The robots trade lines and act the story out. Even the backing singers chant about the story, and every tap of Random brings an episode you've heard least.
  • Music visualization: every mouth shapes the vowel it sings, lyric cards float under each head, and sparkles flare around a robot while it sings. The song grows as you wake more voices.
  • Spatial audio: each robot sings from its own head. Carry one across the room and its voice moves with it.
  • Interactive, at your pace: pinch a sleeping robot to wake it, pinch again to choose who sings what, or type a line and the choir sings it back. Wave, make a heart or a thumbs up, and they answer. Hold out a finger and a robot takes it. Pet, boop or scratch a head and it leans in happily.

Everything works seated, within about arm's reach. First-timers see only the essentials until the song has played to its end. Songs are shared with one QR code.

How we built it

Unity 6 on OpenXR with Meta XR Core SDK v207: hand tracking, passthrough, and a persistent spatial anchor for every robot, so the choir is still where you left it next time. The app is hands-first end to end; picking up controllers shows a friendly "Put your controllers down" panel. Far robots take a hand ray from your shoulder through your thumb and a pinch, so no eye tracking is needed; near ones a direct pinch or a fingertip poke. QR import reads the passthrough camera on Quest 3 and 3S.

The voices are procedural formant singing (no recordings, no text-to-speech), driven by a pronouncing dictionary and rendered ahead on worker threads. The songs come from an engine-independent core that composes each one from a seed, so a whole song, with your edits, fits in a QR code.

Challenges we ran into

  • Sixteen synthesized voices and a frame rate that never dips.
  • Fingers that touch: every finger segment collides with the real head shape, and the head yields and springs back, smoothed so tracking jitter doesn't shake it.
  • Telling a clap or two cupped hands around a robot (which counts as a pinch) from praying hands or a passing wave.
  • Backing chants that tell the story without ever talking over the storyteller.

Accomplishments that we're proud of

  • 90 fps on Quest 3S with sixteen live voices, above the 60 fps minimum.
  • Hands only, from the first second, and no wall of text: instead of a tutorial, a ghost hand shows each move when you need it.
  • Twelve story songs for the whole family, each rewritten until it read better than the last version.
  • A song in a QR code: another headset scans it and plays the same song, with your words.
  • Over a thousand automated tests, and a store release that started as a voice prototype on October 17.

What we learned

Hands need forgiving rules. A closed fist should count as a pinch, a stray touch shouldn't wake a sleeping robot, and every gesture has to tell apart the ones it looks like. Calm is a design goal of its own: nothing to fail, nothing to rush, and the choir keeps singing whatever you do. And mixed reality is at its best when the characters are small and close, in your own room, within reach.

What's next for Soothing Bots

Soothing Bots launches on the Meta Horizon Store in November 2026, free to download. Next: more episodes, more hand plays between the robots, and more ways to see your song grow between sessions.

Built With

  • formant-synthesis
  • hand-tracking
  • music-theory
  • unity
+ 2 more
Share this project:

Updates

Submission history