CrucibleXR

It's exercise meets aliens! CrucibleXR is a fitness app with a boxing bend and an alien cyborg wrapper. Perfect for the fitness fan or boxing hobbyist.

Comment 1

Inspiration

I assume that CrucibleXR was delivered through an interdimensional portal in the back of my freezer. I found a pitch black thumb drive while reaching for yet another pint of Chocolate Chip Cookie Dough on a pit stop during a code sprint. It was just sitting there, neon pink and blue lights pulsing, slowly melting everything around it. So, like any security conscious developer, I grabbed my ice cream and took the thumb drive straight to my laptop to plug it in, anxious to see what goodies I’d found. Almost immediately sparks flashed and streaks of electricity arched across my keyboard. And just like that, my laptop was dead. It was in that instant, when the fried monitor turned shiny black, a perfect mirror of my office, that I saw my own reflection. Still holding a pint of ice cream, spoon dangling between my lips and I thought; Damn. I need to lose a few pounds. So I decided to make a fitness App.And one that would get me back into my old hobby of boxing.

What it does

Crucible XR is a mixed reality exercise app with a sci fi twist. The player selects the number of challenge rounds, the length of those rounds, and their active rest period. Once they’ve placed their hand on the start panel, they're off! Learning boxing combos called out by a Wit voiced bot, trying to hit tiny flying machines hell bent on carving away their real world space, replacing it with a dystopian cyber world, boxing against music driven arrows in an endurance round and finally, being teleported to a full VR battle against flying bots. All this while getting “encouragement” from a snarky interdimensional AI.

How we built it

CrucibleXR was built using Unity as its game engine and the Meta Presence Platform for XR interaction. It uses scene understanding to read a player's previously scanned room model and place objects on their walls as well as have bots interact with them as they attempt to overwrite our world with theirs. It uses Wit.AI for Text To Speech to read boxing combos, give instructions, and mock players with an other-worldly, snarky attitude. Hand tracking is used for all interactions and Hand Poses are used for shooting lasers and starting the game. Spatial audio is used to keep the player on their toes as bots and other objects are spawned around them and they have to use the sound to turn in the right direction for interaction. I really want to create an experience that was both XR and VR so for the majority of the game Passthrough is used for placing objects in the players real world, but full VR takes over for one of the challenges to give a sense of scale and impending doom.

Challenges we ran into

Time is always the greatest challenge in a hackathon. Initially the project was going to have several exercises that made greater use of hand tracking: pushups, climbing, lifting and moving objects to get a wider range of physical motion. But initial quick tests indicated that it would take more time to make these exercises work with the concept and be fun. While I’m quite happy with the initial build I’m looking forward to continuing the game and adding a host of exercises I didn't get around to. On the technical side, I found that incorporating wit.AI into game play adds a bit of lag that slows down game play. I'm looking forward to LLM's that are small enough to be ported on device to avoid this issue.

Accomplishments that we're proud of

Quite happy with getting bots to fly around using the bounds of the room set by the players room setup. It took a bit of digging to find a solution that worked with the current version of the Presence Platform I used but in the end the bots stay reasonably well within the bounds. I’m looking forward to experimenting a bit more and getting their flight patterns to interact with the walls and furniture in a more “natural” way.

What we learned

I always try something new with every project and this one was no exception. One of the best was how easy it is to get started with Meta’s Presence Platform. Integrating aspects of scene understanding, hand tracking, and passthrough were at the start quite easy. Bending them to get them to do what I wanted was a bit more of a challenge but quite a bit of fun. Hunting through the documentation and finding information that was current to my version of unity and the version of the Meta Presence Platform was quite a challenge and gives me a little pause as a developer not knowing what aspects will still work in a few months with the constant rate of updates. In the end this hackathon reinforced how frustrating and fun developing for XR can be, especially with new and improving tools.

What's next for CrucibleXR

According to the online dictionary: Crucible: is a situation of severe trial, or in which different elements interact, leading to the creation of something new. With that in mind, The four exercises I managed to complete during the hackathon are just the start. I plan to explore and add multiple exercises that take advantage of Meta’s hand tracking and room models. I also intend to further gamify the experience, creating leaderboards, music selection, as well as a social component, that with any luck allows players to exercise and battle together, getting up off the couch and earning that pint of Chocolate Chip Cookie Dough.

Share this project:

Updates