Inspiration

Smart-home apps live on phones: unlock, find the app, find the room, find the device. In mixed reality your home can sit in front of you, so we built the shortest path from "living-room lamp on" to the lamp being on: no controller, nothing new to learn, no surprises.

What it does

Turn your left palm towards you and a small model of your home appears on it. Pin it on the table, grab and turn it. Pinch a room and it opens in front of you while you stay seated; pinch the lamp and it turns on. The built-in demo home means judges need no hardware.

Add a device in four steps: pick its type, name it, place it, confirm. Locks, cameras and drones also ask for your PIN (demo PIN: 0000).

Simulated security drones wait at their docks around the house; a geofence keeps every flight over your own property, with privacy towards the neighbours, an altitude cap, a safe link range and automatic return.

Camera feeds open as windows you can move, resize and minimise. They need your PIN, stay on your home network, and close when you stop watching. Energy, water, solar, battery and generator dashboards show the home at a glance.

An agentic assistant is built in with logic and monitoring: on-device rules, no cloud AI. Type an everyday English command such as "lights off in the kitchen" and it knows which room you mean. It always asks before acting and needs your PIN for locks. Voice input is a labelled demo in this build.

How we built it

Unity 6 with the Meta XR SDK and the Mixed Reality Utility Kit: passthrough, hand tracking, scene understanding. It connects to your own open-source home hub on your home network and controls supported smart TVs directly. The core is engine-free C# with 333 automated tests, all passing.

Privacy

Your home stays on your headset. The app talks only to your own home hub and your TV: no recordings, no cloud, no servers of ours. Cameras open only on your home network, behind your PIN.

Challenges

Seated, hands-only input that feels reliable; crisp text at arm's length; a whole home inside the Quest's frame budget; and safety without friction.

Honest note on the demo

The drones and camera feeds are simulated over the demo home; no real drone is connected. Voice is a demo in this build.

What's next

Heater temperature dial (°C · °F) and robot-vacuum cleaning zones (in development), real on-device voice, more TV and device makers, the Matter standard, Arabic and more languages, and a public release on the Meta Horizon Store, planned to launch alongside Meta's VR Glasses.

Built With

  • c#
  • hand-tracking
  • meta-quest-3s
  • meta-xr-sdk
  • mixed-reality-utility-kit
  • passthrough
  • unity-6
Share this project:

Updates

Submission history