Stack Lab

A hand-tracking puzzle game where you help a robot named Matty learn the art of human balance and problem solving. Use your hands to stack shapes on a small platform while Matty observes and reacts.

Comment 1

Inspiration

Stack Lab started as a simple question: What would teaching a robot spatial reasoning actually feel like? I imagined something small and tactile, more like playing with toys on your desk than exploring a giant world. Just a lab table, your hands, and a curious robot named Matty watching you work

The stacking mechanic grew out of the idea of physical puzzle tasks that are trivial for humans but surprisingly hard to reason about computationally for AI. Balancing odd shapes, feeling weight and friction, and making judgment calls about the space.

What it does

Stack Lab is a seated, hand-tracking puzzle game for Meta Horizon OS.

You sit in front of a virtual lab table (optionally aligned to your real desk). In the center is a small platform. Matty, a friendly robot in training, sits across from you to learn how humans solve and balance problems. He will comment on your progress and skills.

Using just your hands, you grab, rotate, and place physics-driven pieces onto the platform. Some are heavy and grippy, some are light and slippery. When you have satisfied the required number of pieces, you hit a button on the table to verify your stack.

The platform lifts and moves. If the tower holds and all required pieces stay on, you pass the level. If it collapses, you both learn something, and Matty has opinions.

The result is a calm but tense casual loop: think Tetris meets Jenga, driven entirely by hand interaction and character feedback.

How we built it

Stack Lab is built in Unity for Meta Horizon OS, targeting Quest devices. It uses the Meta XR SDK for hand tracking and interactions, with a hand-first design from the start.

On the technical side:

  • Physics uses primitive colliders (boxes) for reliable, performant stacking.
  • Grabbing is implemented as a physics based attachment so pieces feel like they have real weight instead of teleporting to the hand.
  • Visually, the art direction intentionally mirrors an optimistic future look with clean architecture, blue skies, greenery, and soft organic materials and shapes.
  • The game is tuned to run at a minimum of 60 FPS on Quest 3/3S, with Quest 2 used as the development benchmark.

Challenges we ran into

  • Scope creep. It’s hard not to want to build countless features, the concept can hold quite a lot more than what made it into this vertical slice. Choosing what to cut was its own design challenge.
  • Making the grabbing feel weighty without turning the player into an unstoppable wrecking ball that nukes every stack took a lot of tuning.

Accomplishments that we're proud of

  • Striking a good balance between fun and engaging content. Easy enough for anyone to pick up but a great challenege for those that want to dive deeper and reach the very high levels.
  • The moment when players instinctively rest their hands on their real table and the virtual desk lines up feels surprisingly magical.
  • Hand grabs that feel “soft” and physical, not robotic.
  • A tight vertical slice that already feels like a small but polished casual experience.
  • Matty’s personality, tiny lines of dialogue that make a very simple setup feel alive.

What we learned

  • Working on Stack Lab reinforced how much small interaction details matter in VR, especially with hands: collider shapes, force limits, and visual feedback. Sound design and feedback are incredibly important when you’re missing the haptics of controllers. Using a real world table as an anchor enhances comfort, presence, and clarity.

What's next for Stack Lab

  • Turning it into a social experience where more than one player can attempt to build a stack together.
  • Adding “special pieces” that I had to cut from this slice. For example, a piece with a fire property where flammable materials can’t be placed next to it, or fragile pieces that can’t support heavy ones. The interactions could be endless, and I’d love to push the physics systems further.
  • Adding achievements, challenges, and leaderboards to connect players around solving these puzzles in different ways.
  • Gathering player feedback to improve comfort, difficulty tuning, and accessibility for a broader audience.

Built With

Share this project:

Updates