Inspiration

The original idea of Hander Attack comes from the concept of the digital twin. A digital twin creates a live, data-driven mirror of a physical space, allowing you to monitor, analyse, or even control what happens there. We translated this idea into gameplay through a simplified version of a digital twin: an interactable minimap of a room. Traditionally, minimaps only help you navigate the world, not influence it. In Hander Attack, the minimap becomes the controller, exploring how interacting with a representation of a space can affect the real world.

What it does

Hander Attack is a mixed reality, hand-tracking multiplayer game for Meta Quest where your own room becomes the arena. First, the app scans your room and generates a minimap of it. Then, both minimaps are swapped, giving each player a miniature representation of the opponent’s room. Small interactions on the minimap have immediate consequences in the mixed-reality space (and vice versa), turning it into a real-time chase where one player has to hide while the other tries to catch them. The VFX and audio react to events in the room, reinforcing the immersion, social presence, and the fun of the game! The result is a playful bird's-eye view experience where you try to find and catch the other player hiding in their room using their minimap.

How we built it

We built the project using Unity 6, C#, and Universal Render Pipeline for the rendering, gameplay logic and all interaction systems. For room scanning, we relied on the Meta MR Utility Kit, which provided us with a JSON representation of the environment. This representation was then used to reconstruct the minimap and collision geometry.

Additional core features such as basic hand tracking and pose recognition were implemented using the Meta Core SDK, and Photon was used to explore multiplayer capabilities. For 3D models and animations, we used Blender along with Unity’s timeline and Director tools to handle the full asset production pipeline. Hand tracking was introduced using the Meta MR Utility Kit and we implemented pose recognition. For optimized visual effects that collide with Unity GameObjects we used VFX Graph and ParticleEffects for GPU-enabled visual effects that interact with virtual and physical geometry.

Together, these tools enabled the full flow: room scanning, JSON generation, digital twin reconstruction, minimap syncing, and real-time MR gameplay where players alternate roles.

Challenges we ran into

Making it feel like it’s “happening in front of you”

Because the experience is fully based on hand tracking, we couldn’t rely on controllers or haptics to sell impact. We had to design VFX, audio, and timing very carefully so that every interaction looks and sounds punchy enough to feel physical, even without vibration.

Synchronizing room scans between players

On the multiplayer side, the most difficult part was synchronizing the scanned minimaps between players. After a room scan, the device generates a JSON file, so we rebuilt the room layout from that JSON and used it as the shared reference. We also needed a reliable way to send controller positions: we first converted the controller’s world position into the minimap’s local space, transmitted that data, and then reconstructed the correct world position on the other player’s device. Converting positions between world space and the minimap’s local space, sending them over the network, and reconstructing them so both players see consistent networked interactions in real time was challenging.

Optimising VFX for standalone hardware

We wanted rich, space-aware particle effects, but Quest hardware has strict limits. Our major challenge regarding VFX was to ensure particle systems and shaders were optimized for a stand-alone app, which meant having to do lots of tweaks, such as spawn rate.

Accomplishments that we're proud of

We are proud of implementing a functioning, fun, and aesthetic game that integrates an end-to-end pipeline from conception to the delivery of a polished prototype. We are proud of how clearly Hander Attack communicates the idea of interacting with a digital twin through a minimap while still being entertaining and engaging.

What we learned

Through this project, we learned a lot about how powerful a minimap can be as an interaction tool in mixed reality. It was especially interesting to see how small actions on the minimap could have an immediate impact in another player’s room in real time, creating a strong sense of shared presence.

We also gained a deeper understanding of the design constraints of hand-tracking–only experiences. Without haptics, every piece of feedback has to be conveyed through visuals, sound, and timing. This forced us to think more carefully about how to communicate impact, intention, and responsiveness through VFX and audio alone. Finally, working with room scanning, JSON-based reconstruction, and synchronizing local and world spaces across devices taught us a lot about the technical and UX challenges of building robust multiplayer MR experiences.

What's next for [B04] Hander Attack

Looking ahead, the idea of interacting with distant spaces through a minimap opens up a wide range of possibilities. A small virtual model of a remote environment can act as a powerful control interface, allowing users to manipulate objects, guide other people, or coordinate shared tasks without physically being there. This approach could be extended far beyond simple room interactions: remote assistance, collaborative design, education, and even large-scale simulations all become more intuitive when the minimap itself becomes the main tool for shaping the world.

By refining the accuracy of room reconstruction and improving cross-device synchronization, we imagine a future where users can seamlessly interact with completely different locations as if they were miniature worlds resting in the palm of their hand. This project gave us a glimpse of that potential, and we’re excited to explore how minimap-based interaction can evolve into a new form of real-time, spatial communication.