Inspiration

The core concept of this project comes from Meta VR Glasses, a headset light enough to carry anywhere. I wanted to build something players would come back to every day, and to imagine a future where everybody has a headset with them wherever they go.

Gameplay Design

The scanner in the field and the lab worktable

You play a field agent of FLORA, the Field Lab for Other-Realm Affairs. The game has two parts: a scanner that lets you collect creatures and seeds in the real world, and a lab where you compose your own eco-sphere and watch the miniature ecosystem inside it.

Designed for everywhere

Designed for everywhere

The scanner works anywhere, and different places give different seeds and creatures. A real-time mesh scan shows you the geometry of the place without any room setup, so you can start right away in stationary mode. An on-device recognition model reads what kind of place you are in and what you are aiming at. Because the game can tell a kitchen from a park, different places really do give different finds, and there is a reason to take the scanner somewhere new.

Designed for coming back

An eco-sphere growing during the 60-second run

The lab is inspired by idle games: you place your seeds, compose your eco-sphere, and wait days for it to grow. In the original concept this brings players back each day to see how their eco-sphere has grown. For this demo we wanted you to see the final result, so the process is sped up into a 60-second run.

Embracing generative (algorithmic) art

Procedurally generated plants and creatures

I am a generative artist (algorithmic, not AI-generated), and I love how rules and algorithms create variety and bring an artwork to life. Every plant and creature in the game is procedurally generated. To me, FLORA is a generative artwork in the form of an MR game.

Project Highlights

Real-time mesh scan

Real-time mesh scan

FLORA builds on UXR.QuestMeshing, an open-source package by Udayshankar Ravikumar, extended with steadier surface fusion. The Quest's depth map is fused into a volume on the GPU and rebuilt as a mesh while you look around. You feel you are scanning real surfaces with no pre-scan, which is what makes playing anywhere practical.

Real-time environment context analysis

The six attributes read from two different environments

The passthrough camera image is scaled down and passed to TinyCLIP, a small image-recognition model running entirely on the device. It classifies the place and the object you are aiming at into six attributes: Flora, Aqua, Stone, Spark, Light and Dark. These decide which species are likely to turn up, so a different room, street or time of day changes what you can find.

Ecosystem design

Ground layer, plant layer and creature layer

FLORA has three layers of life. The ground layer: mosses and lichens that creep across soil and rocks, giving the ecosystem its base texture. The plant layer: flowers, ferns and stemmed plants that draw lines through the scene. The creature layer: small animals that add points of light and motion, and the feeling that the eco-sphere is alive.

This follows a basic principle of visual art, point, line and plane, blended with ecology. In total there are 54 organisms: 40 plants and mosses, and 14 creatures.

Real-time terrain baking

The landscape is locked and baked, then plants grow along it

You arrange the landscape yourself: place rocks and branches wherever you like, and the eco-sphere is baked at runtime around your layout. Plants grow along the terrain you built, and creatures walk, climb and hop across it. The scene is baked into a low-resolution 3D texture that the shaders read for lighting, so the eco-sphere has soft shadows and atmosphere while performance stays high on a standalone headset.

Optimised for hand tracking

Hand poses: glove, tweezers, scanner, worktable

The whole game flow is designed hands-first, with gestures such as putting on the gloves, picking up the tweezers, holding the scanner and carrying the worktable. This lets the player feel the interactions are intuitive and natural throughout the entire game.

Putting on the glove

Some of the items were also redesigned many times to get the experience right. Take the scanner: at first it had a pistol-grip design with an index-finger trigger, but the headset's cameras have a hard time seeing the index finger in that position, so the trigger was very hard to pull. Due to this issue, we redesigned the scanner four times, trying out different holding poses, before arriving at the current design, which finally feels right.

Scanner iterations

Looking forward to the future

As an artist who focuses on visual form, I am super excited that VR devices are getting lighter while keeping their computing power. Today's hardware is already strong enough for beautiful visuals, and I look forward to a future where it is much easier for people to experience my art in an immersive 3D environment.

Future plans for the project

The project is a proof of concept that I am happy with, especially since it has only been in development for about two months.

If the project is going to ship, the ecosystem design needs to be refined: the life inside the eco-sphere should respond to and influence each other more, so players feel the joy of cultivating. It also needs a way to share, since it is a casual game about creating.

I will keep showing it and looking for collaborators. With a designer or a plant specialist to complete the remaining parts, I would be very interested in releasing it on the store.

Authorship and use of AI

FLORA is a work shaped by my own aesthetic and ideas. It is also a deliberate experiment with a modern AI-agent production pipeline.

Most of the implementation was done by Piyellow, my OpenClaw coding agent, under my close direction. I sent it more than 100,000 words of direction, about the length of a novel.

The code integrates several open-source projects: UXR.QuestMeshing (Apache-2.0; adapts lasertag and naive-surface-nets, MIT), TinyCLIP (MIT), the CHIMERA vocal synth geometry for the guide's hologram head (MIT), and the Saira Condensed font (OFL).

For the art, I start from hand-drawn sketches, refine the look with ChatGPT, and then hand it to Piyellow to model procedurally in Blender, usually with a very precise description of the geometry.

The music was made with Suno and edited by Piyellow, the guide's voice uses OpenAI text-to-speech, and the sound effects are synthesized by scripts.

Built With

Share this project:

Updates

Submission history