Inspiration
Most voice experiences treat conversation like a search engine: you ask a question, and it recites an answer. We wanted to flip the script and turn conversation into play. What if you hold the secret, and Sherlock Holmes and Dr. Watson have to deduce it by questioning you?
The spark came from the smallest mysteries of everyday life—where on earth did I leave my keys? Holmes and Watson are the perfect pair to treat that kind of domestic nonsense as a matter of utmost Victorian gravity. TruSleuths is a warm, witty, seated mixed reality experience built around the simplest controller in the world: your voice.
Behind the project: TruHealer
TruSleuths comes from TruHealer, a healthcare training startup building the next generation of clinical simulation. TruHealer helps train better doctors and nurses using Extended Reality and hyper-realistic conversational patients—letting learners sit face-to-face with a lifelike virtual patient, completely hands-free, to practise the conversations that matter most before they happen in real life.
Because our clinical platform is already built around natural, hands-only interaction in mixed reality, this competition was a natural opportunity to bring our technology to a wider audience and show what Meta Quest can do beyond traditional gaming. We took the same core stack—photorealistic MetaHuman characters, natural speech, a real-time conversation engine, and Quest passthrough—and pointed it at pure entertainment: two legendary detectives and a missing set of keys.
If a lifelike character can make you forget you are wearing a headset while you invent an alibi in your living room, that same presence can prepare a nurse for a difficult bedside conversation. We hope TruSleuths shares the joy of talking to believable virtual humans—while showing how headsets like Quest can make the world a better place, starting with how we train the people who care for us.
What it does
Put on your headset and Holmes and Watson sit right across from you in your actual room, with lifelike faces, voices, and eye contact. Tap Generate story on your in-world clipboard for a lighthearted mystery prompt (or bring your own), keep the answer up your sleeve, and ad lib. Holmes probes with one sharp question at a time, Watson plays along with warm curiosity, and if you blurt out the whole solution too early, Holmes looks wonderfully wounded that you spoiled the chase. Pass the case between detectives whenever you like, or wrap it up when you are done.
- 100% hands-only, no controllers needed: Speak naturally, tap with a finger, or raise an open palm to interrupt mid-sentence.
- True living-room mixed reality: The detectives share your physical space rather than pulling you into a closed virtual world.
- Made for sitting back: A relaxed, seated format that is effortless to pick up, play for a few minutes, and pass around the sofa.
- Clever, gentle comedy: No timers, no scores, and no wrong answers—just deadpan seriousness applied to your everyday absurdities.
How we built it
- Unreal Engine 5.8 (C++ and Blueprints) for Meta Quest 3 and 3S, using the Meta XR and OpenXR plugins for high-presence passthrough and hand tracking.
- Custom-tuned MetaHuman characters driving real-time lip sync, eye contact, head-gaze tracking, and context-aware idle, listening, and thinking animations.
- A cloud back end on AWS that orchestrates dialogue, story generation, and voice synthesis over a low-latency WebSocket—keeping the headset client lightweight and free of embedded secrets.
- Full Meta Horizon platform integration with seamless sign-in, native subscriptions, add-on hour packs, a free 7-day trial, and fair usage metering that counts time only while a detective is actively speaking.
- Tactile, world-space UI: A notebook-style clipboard for story prompts, detective handoffs, and remaining time, paired with a clean status indicator that always shows when the app is ready, listening, or thinking.
Challenges we ran into
- Desktop-grade humans on mobile XR. MetaHumans are built for high-end PCs. Keeping two detailed characters sharp and comfortable at Quest frame rates meant carefully optimising hair cards, LODs, materials, and animation without losing the facial nuance that sells the illusion.
- Making voice turn-taking feel human. In conversational XR, latency and pacing decide whether you feel like you are talking to a person or waiting on a server. We built an in-character idle ladder, a palm-raise interrupt gesture, and clear visual cues so every beat stays alive.
- Store-ready polish. Building for the Meta Horizon Store meant going well beyond a hackathon prototype—handling focus-loss pausing, offline recovery, permissions, custom splash screens, and data-use disclosures. Because the Quest 3S has no proximity sensor to detect when a headset is set down, we protected players' balances with idle timeouts and by metering active detective speech only.
- Privacy by design. We built the session flow around anonymous platform account IDs only, backed by a clear privacy policy, a 13+ audience rating, and on-request data deletion.
Accomplishments that we're proud of
- Believable characters in your real room. Getting photorealistic MetaHuman detectives running smoothly in Quest 3 and 3S passthrough—complete with natural eye contact, live lip sync, and Holmes's theatrical disappointment when you spoil the ending.
- Zero-controller conversation. Designing an experience you can play from start to finish with nothing but your voice and your bare hands.
- Shipping a complete store release, not just a prototype. Passing Meta's Virtual Reality Checks and integrating full platform sign-in, subscriptions, a free trial, and fair speech-only metering.
- One technology, two worlds. Proving that the same TruHealer conversational human stack built for serious healthcare training can cross over into warm, accessible living-room entertainment.
What we learned
Great hands-only voice design is really about active listening: people will happily wait while a character ponders, shifts posture, or raises an eyebrow—what breaks the spell is silence. Subtle visual cues and presence matter far more than adding another menu. We also saw firsthand how much craft sits between a working prototype and a store-ready Quest release, and why nothing replaces testing on a worn headset in a real room.
What's next for TruSleuths
Expanding the casebook with new mystery packs and seasonal prompts, richer two-detective banter, and bringing TruSleuths—and the TruHealer conversational human platform behind it—to more people and upcoming XR devices.
Built With
- amazon-web-services
- azure
- blueprint
- c++
- hand-tracking
- meta-horizon
- meta-quest
- mixed-reality
- openxr
- unreal-engine