Table Space: Mixed Reality Tabletop Sandbox

Mixed Reality Tabletop Games Sandbox: Play and Create your own Board Games, Trading Card Games, and more in your physical space. Major update with Meta Avatars and experimental MRUK QR code scanning.

Comment 1

Inspiration

We grew up playing countless rounds of trading card games and board games with friends and family. With this frontier of mixed reality, we have an opportunity to reimagine these experiences and allow for seamless multiplayer regardless of where you are in the world, while retaining that sense of magic, self-expression, and human-connection.

What it does

Table Space is a tabletop games sandbox that lets you create and play tabletop games with friends in mixed reality on Meta Quest, and feel like you're in the same room together with expressive Meta Avatar telepresence. Sit across from the virtual tabletop and play classics like Chess or Checkers, or instantly load up your trading card game decks with the new experimental QR code scanning support via the Boxtree Builder companion website.

How we built it

We used Unity for the core game client, Meta's MRUK for the new experimental QR code scanning capabilities, Normcore Realtime for multiplayer networking capabilities and our new first-class Meta Avatars integration.

We also built a companion web application, the Boxtree Builder, to load up custom card/board game configuration with TypeScript on the frontend and Elixir with Phoenix on the backend (https://builder.boxtree.gg).

Challenges we ran into

We had some initial stumbling blocks learning the ins-and-outs of MRUK, Meta Avatars (the distinctions between eg 1st person and 3rd person avatar variants), multiplayer networking code, passthrough camera access.

Also had the challenge to convert a camera tracked position onto a world space position. EnvironmentRaycastManager.Raycast() method helped us a lot to get it just right.

Accomplishments that we're proud of

We're delighted by the smooth Meta Avatars interaction experience: it feels really magical to sit across the table from a friend in your physical space and feel like they're there in the room with you.

We also are so thrilled by the feeling of acceleration that QR code scanning offers inside Table Space: now you can effortlessly jump into the fun part of playing eg Trading Card Games or testing out your new card game configuration from our web-based builder tool.

We went even further and added multiple April Tags tracking to real life physical playing cards. Now we can blur the distinction between virtual and real world card and board games. Simply print and paste these tags to card sleeves or cheap playing cards to get them augmented with anything you want. This is all supported thanks to our custom game builder tool, players can overlay their cards with 3D models and images if they so desire. April Tags Demo video: https://youtu.be/rsh91VWWCH4 [Not yet on the release channel due to it still being unstable, but can share a binary if needed]

What we learned

We learned a ton about Passthrough Camera Access and the full suite of capabilities offered by MRUK.

What's next for Table Space: Mixed Reality Tabletop Sandbox

We're excited to add an effortless physical table-anchoring solution using MRUK and the new environmental raycasting capabilities offered in the latest horizonOS.

We've also been experimenting with the Passthrough Camera Access API to do some really cool hybrid physical-digital interactions taking advantage of fast-tracked fiducial markers (ie AprilTags) to use physical objects in your space (paper cards, board game tokens) to control virtual objects and interactions in-game with 3D dynamic overlays and effectively "teleport" physical objects to bring them into the embodied telepresence experience with friends.

We're now in the midst of creating our own Trading Card Game concept dubbed Spatial Creatures to showcase these capabilities, where you can spawn creatures from physical or virtual cards, collect packs, with characters stereo-rendered in delightful 3D inside the immersive card, and watch them come to life and interact with your environment when you place them down on the tabletop.

Here's a video clip highlighting all of these planned capabilities which are under active development: https://www.youtube.com/watch?v=e-_WHwoWZRk

Built With

Share this project:

Updates