I want to turn Earth into a world of exploration again.

For most of my life, the biggest adventures in games happened while I sat still, trapped in one room and tied to a PC. The worlds were huge, but I was never really in them. Mixed reality made me ask a new question: what if the adventure came to us instead?

What if there were a parallel world, Altera, layered over our own, that you could only see and reach through a mixed reality headset? Your living room, your backyard and your town would become places to explore, with creatures to find, bond with and raise.

Soul Bearers Altera: Ethereal Pets is the first step. It starts at home, with a pet that lives in your real space, knows your voice and grows with you. Its twin game, Altera Earth, carries the idea out into the world as a boundless MMMR RPG (Massively Multiplayer Mixed Reality Role-Playing Game), where you are the character and use your own body to explore, gather, craft and fight.

What I learned

  • A pet only feels alive when it chooses for itself. Scripted animations looked fake. A utility AI that weighs each pet's personality traits (bravery, laziness, playfulness, loyalty and more) made Bambi feel like a real companion.
  • Mixed reality is physics first. A pet that sinks into the floor or floats above it breaks the illusion instantly. The room scan, gravity and floor detection have to be right before anything else matters.
  • 3D animation has rules you only learn by breaking them. Bones and shape keys don't simply add together, and a model that looks perfect in Blender can behave very differently in the game.
  • Ship tools, not just features. Building my own creator admin, updater and Blender scripts means I can fix things myself instead of starting over.

How I built it

  • Engine: Kotlin on Meta's Spatial SDK for Meta Quest 3, using passthrough, room scanning (MRUK), hand tracking and voice input.
  • Creatures: modeled and rigged in Blender, exported as glTF. Each pet has skeletal animation clips (walk, run, sit, lie down) plus shape keys for the face and body (panting, head turns, body bends).
  • Living animation: my own animation layer plays bone clips and shape keys together, and keeps separate parts like the eyes, teeth and tongue attached to the body.
  • Pet brains: each pet has a kingdom, an affinity, stats and a 16-trait personality. Stats and personality are rolled when you receive a pet, so no two are the same, and a utility AI picks what the pet does next.
  • Interaction: voice commands (come, follow, sit, paw, roll over, play dead...), pointing to send your pet somewhere, and a "What can I say?" panel that follows you and scrolls when you point at it.
  • Server: a PHP backend and web admin where I build each creature (model, icon, stats, personality), plus an in-game updater with debug, tester and release channels, so new pets and fixes arrive without reinstalling.

Challenges I faced

Bones vs. shape keys. In glTF, shape keys are applied before the skeleton, so a bone always gets the last word. When Bambi's body bent to look at you while a running clip played, the leg bones flung his paw half a meter away. I solved it by keeping the legs out of the bend and letting the clips own them.

Head turns that broke the head. Turning the head and turning the body were separate shape keys. Adding them together made the head grow to over 150% of its size. The fix was to cross-fade them instead of adding them, where $b$ is how far the body has turned:

$$ w_{\text{head}} = 1 - b, \qquad w_{\text{body}} = b $$

I then generated new shape keys so he can still look up and down while his body is turned.

Parts flying off. The eyes, teeth and tongue are separate meshes, and every shape key has to exist on each part. A single misnamed key (Turn-head_left instead of Turn_head_left) or a wrong shape made an eye or the teeth fly away when he panted. I wrote Blender scripts that measure how the head moves and rebuild the matching keys on every part automatically.

Staying on the floor. Pets sank into the floor or fell forever. Birds tried to take off from under the ground. One line that skipped flying pets with return instead of continue stopped every pet after the bird from being grounded. Small bugs in mixed reality are very visible.

Updates over bad Wi-Fi. Testing on a throttled hotel network, the headset kept loading old models because slow downloads timed out. I learned to trace the whole update path, from upload to server to headset, and to check what actually lands on the device.

What's next

More creatures across every kingdom, pet battles, evolution, a companion app, and the link to Altera Earth, so a soul you raise at home can follow you out into the wider world of Altera. ## See my progress

You can follow my development progress on Meta platforms (Instagram):

Built With

  • metaspatialsdk
Share this project:

Updates

Submission history