Inspiration
Phones “zoom” by cheating — cropping, scaling, stretching pixels. Now that Meta’s PCA API finally lets us touch raw camera textures, we wanted to see how far we could push that idea inside mixed reality. We explored everything from micro-civilizations living on leaves to biology-scale deep dives into cells. Eventually, we pivoted into something more playful: a zoom mechanic that reveals supernatural layers hiding in your actual room. Think Luigi’s Mansion, but your hands are the controller and your environment is the level.
What it does
Normally, your real world is stuck on one layer. With ZOOMR, you peel that limitation away. A left-hand gesture “zooms” through multiple invisible dimensions around you, revealing ghosts tucked inside different layers. Your right hand does the cleanup — form the gesture, shoot, and the spirits vanish.
How we built it
We built everything in Unity using the Meta XR SDK and passthrough features, stitching together hand-tracking, custom interactions, and our own zoom-through-reality pipeline. Raw camera textures come straight from the PCA API; our ghost lenses and detector objects were modeled in Maya. The whole system runs on lightweight cropping/scaling logic so we keep full control over digital zoom without relying on AI tricks.
Challenges we ran into
Our original plan involved a fully custom hand-gun gesture for raycasting, but designing, training, and tuning that in hackathon time was ambitious. We kept the silhouette but simplified input to a thumb-and-index pinch — fast to implement, still readable, and reliable under hand-tracking jitter.
Accomplishments that we're proud of
We’re proud to be using the PCA API the way it was meant to be used — directly manipulating raw camera textures to create a believable zoom mechanic. No enhancement models, just pure control. We also got MR animations, audio cues, and object spawning working cleanly, including automatic ghost respawning to keep gameplay active.
What we learned
We dove deep into the PCA API, MRUK, and how far passthrough can be pushed when you stop thinking of it as a background and start treating it as a layer stack you can manipulate.
What's next for ZOOMR
On the tech side, the zoom mechanic could expand into accessibility tools — magnification, selective enhancement, or guided vision overlays. For the game, eye-tracking could move the lens itself, freeing the player’s hands entirely. And we want to evolve the ghosts — color-coded by depth layer, more expressive behavior, and richer feedback to make each resolution feel like its own world.
Log in or sign up for Devpost to join the conversation.