Wall Breaker MR

Inspiration

We were inspired by the classic "Breakout" game genre. We thought, "What if we took this simple, addictive mechanic and brought it into 3D Mixed Reality?" We wanted to see how much more stimulating and physical the experience would become when the target isn't just blocks, but your own real-world environment.

What it does

Wall Breaker MR is a physical action game played in your real room.

Scan: The app scans your physical environment (walls, ceiling, furniture) to create a destructible mesh.
Create & Destroy: You use a "Circle" hand gesture (forming a shape with both palms) to generate physics-based spheres. You then launch them to smash the walls in front of you.
Reveal: As the physical walls break, a beautiful 3D skybox is revealed behind them.
Magic Moment: The visceral experience where not just the walls, but even your real furniture appears to crumble, blending the destruction of reality with a digital sunset.

How we built it

We used Unity and the Meta Presence Platform.

Scene Understanding (Scene Mesh): We utilized the Scene API to generate a mesh collider that perfectly matches the user's physical room. This allows the balls to interact physically with real walls and furniture.
Interaction SDK: We utilized Hand Pose Detection (Finger Features) combined with custom logic to detect when both hands form a "circle" shape to spawn spheres. We also used Poke Interactions for palm-based UI menus.
Passthrough & Masking: We implemented custom shader logic to handle depth sorting, ensuring the digital "Skybox" renders behind the Passthrough layer only where the wall is broken.

Challenges we ran into

Rendering Order: We struggled to control the drawing order of multiple object types (Passthrough, opaque walls, transparent effects, and the Skybox). Ensuring they layered correctly was our biggest technical blocker.
Hand UI: UI panels kept clipping into physical walls. We wrote a script to dynamically resize and offset canvases based on wall proximity.

Accomplishments that we're proud of

Achieving the visual illusion of "depth" behind a solid physical wall.
Creating a satisfying physics-based loop without controllers.

What we learned

We learned that in MR physics games, purely physical interactions aren't enough. Visual and audio feedback (like dust effects and impact sounds) are critical to making the destruction feel "real" and satisfying.

What's next

Difficulty Progression: Implementing level-based adjustments to keep the challenge engaging.
Surprising Gimmicks: Adding interactive elements and events that entice players to break more walls to see what lies in the outside world.
Social Connection: A feature to leave "messages" or "graffiti" on the virtual skybox for other players to find after breaking their own walls.