MR Hand Hygiene learning experience

A mixed-reality, hand hygiene trainer for Quest 3/3S based on the WHO technique: a virtual sink, soap and towel placed in the learner's real room, nine guided steps, voice or button navigation.

Comment

Inspiration

Hand hygiene comes before everything else in healthcare, food service and hospitality. It is the first thing taught before PPE, and also the first thing people get lazy about. The WHO technique has several steps, and most people don't know the full sequence. Posters on a wall and slide decks don't let anyone practise it.

I wanted a trainer that works in the real room, at the real sink, with your own hands: calm, simple, and easy for any organisation to add to the training it already runs.

What it does

MR Hand Hygiene is a mixed-reality trainer for Meta Quest 3 and 3S. Using passthrough, it places a virtual sink, soap dispenser and paper towel dispenser over your real sink. A guide then walks you through the nine WHO handwashing steps.

  • Animated ghost hands demonstrate each step, and you copy them with your own tracked hands. There are no controllers.
  • Narration reads each step aloud.
  • You say "Next" to advance, or press the Next button. Voice is optional and the button always works.
  • You position the virtual guide by grabbing it and moving it next to the real sink.
  • An admin can read a simple session log from the headset (date, duration, completed or not, steps reached). It stores no names.
  • The whole experience also runs without Wi-Fi, using the Next button for navigation.

How it built it

  • Engine and SDK: Unity 6 (6000.3), Meta XR All-in-One SDK 205, OpenXR, passthrough, hand tracking and the Interaction SDK for grabbing.
  • Flow: one controller script (PanelFlowManager) manages every panel, the guide placement, grab state and step progression. Voice and button input both go through the same step-advance path.
  • Content: the nine steps live in a JSON file (text, timing and audio mapping), so instructions can be changed without touching code.
  • Voice: Meta Voice SDK with Wit.ai recognises "Next".
  • Narration: nine pre-recorded audio clips, one per step.
  • Hands and fixtures: the sink, soap dispenser, towel dispenser and ghost-hand poses and animations were made in Blender and imported into Unity.
  • Logging: a lightweight CSV session log on the device.
  • Configuration: nearly everything (timings, placement distances, heights, logging, voice on or off) is editable in the Unity Inspector, so a client can adapt it without editing code.

Challenges we ran into

  • Invisible panels on Quest. The UI looked fine in the Unity Editor but was invisible on the headset. The overlay canvases were using a composition mode that fights with passthrough. Switching them to Depth Tested fixed it.
  • Voice input. I tried running speech recognition fully on the device, but on Quest 3S it took 11 to 21 seconds per command and was often wrong. A commercial on-device engine turned out to be priced far beyond a solo project. I settled on Wit.ai and kept the manual Next button as a fallback, so training never depends on the network.
  • Voice only half-working. The headset heard "Next" but the panel didn't advance. The fix was to listen to the full transcription event and match the whole word, with a short de-duplication window.
  • Testing without hardware. The XR simulator can't render passthrough, so passthrough had to be checked on a rented Quest. I built the logic in the Editor first and verified the experience on the headset afterwards.

Accomplishments that we're proud of

  • A complete, working nine-step WHO handwashing flow, tested end to end on a Quest 3, with voice and without Wi-Fi.
  • A release-candidate build of about 80 MB.
  • A design that is easy for others to adapt: change the text, audio, timings and placement without writing code.
  • Making a calm, practical training tool that runs in a real room instead of a simulated one.

What we learned

  • Passthrough, overlay UI and the Editor all behave differently, so only the headset tells you the truth.
  • A fallback that always works (the manual Next button) matters more than a perfect voice feature.
  • Keeping one authoritative controller for the flow made bugs much easier to find.
  • Sharing the demo early and writing down the white-label options brought in conversations I wouldn't have had otherwise.

What's next for MR Hand Hygiene learning experience

  • Add more languages and narration.
  • Explore on-device keyword spotting so voice works fully offline.
  • Extend the same format to related topics, such as putting on and removing PPE.
  • Talk to healthcare and training teams about adding it to their existing content, and look at other headsets such as PICO.

Built With

Share this project:

Updates

Submission history