Inspiration
A friend told me about the Meta VR Start competition, and I started thinking about what kind of experience I could create around hand tracking.
I currently live in Georgia, where there's a lot of beautiful greenery. One day, while walking outside and looking at the plants around me, I had a thought: wouldn't it be cool to grow something like this right in my own home, just by touching a surface?
At first, I imagined growing a single plant. But why stop there? What if you could turn your entire room into a garden, using nothing but your hands?
That's how Dryad started. I wanted to create something simple, intuitive, and satisfying. No complicated menus or tools. Just your hands and a world that responds to them.
What it does
Dryad is a mixed reality sandbox where you can transform your surroundings into a living garden.
Touch a table, shelf, or another surface, and grass grows beneath your fingers. On walls, you can grow vines. From grassy areas, you can pull flowers and trees out of the ground, grow new branches, and shape plants with your hands.
And if you don't like something, you can change or remove it. Rub your hands together to create fire and burn unwanted vegetation, or chop trees using the edge of your palm.
Everything is controlled through natural, context-sensitive hand gestures, without unnecessary menus.
How I built it
I developed Dryad in Unity for Meta Quest 3 and 3S, using hand tracking and procedural geometry.
Trees grow along the movement of your hand. Their trunks, branches, and UVs are generated procedurally using PolySkeletonMesh. When you pull a new branch from a tree, it becomes part of the same mesh, creating a continuous surface.
I also developed shaders that let grass bend under your hands and vines react naturally to touch. Grass remembers the direction of your movement before slowly returning to its original position.
Gesture recognition was another important part. I combined Meta's hand tracking data with finger distances, hand poses, and movement detection to make interactions more reliable.
For Quest performance, I used GPU instancing for grass and spread heavier procedural mesh operations across multiple frames.
Challenges we ran into
The biggest challenge was actually finding enough time to work on the project!
Making gestures reliable and intuitive took a lot of experimentation. Procedural geometry also brought plenty of unexpected bugs, some of which still need fixing.
As a technical artist, I already had experience with real-time graphics and optimization, so performance was relatively manageable.
I had many more ideas than I could implement in the available time. Some already have models, sounds, or early technical work behind them, but I had to leave them out of this prototype to focus on the core experience.
Accomplishments that we're proud of
I'm especially proud that I managed to turn the original idea into a working mixed reality prototype where you can actually grow and shape plants with your own hands.
Building procedural trees with editable branches, making vegetation respond to touch, and connecting all of this through hand gestures was a really interesting technical challenge.
What I like most is that all these systems work together to create something that feels simple and natural to the player.
What we learned
One of the biggest lessons was that making hand interactions feel natural is much harder than it looks. Even simple gestures require careful detection and a lot of testing to work reliably.
I also learned how important it is to keep the scope realistic. It's very easy to come up with dozens of exciting ideas, but much harder to find the time to implement all of them.
And sometimes, polishing a few core mechanics is much more valuable than trying to add everything at once.
What's next for Dryad
My biggest goal is to turn Dryad into a living ecosystem that can develop on its own. Plants and creatures would interact with each other, react to their environment, and create a world that feels alive even without the player's involvement.
I'd love to add many more plants and creatures: bees, spiders, snails, and many, many others. Each would have its own behavior and role in the ecosystem. I also want to introduce harmful plants, pests, and other challenges, including ways to deal with them, cooperate with them, or perhaps even tame some creatures.
But most importantly, I want every plant, creature, and object to offer multiple ways to interact with it. Everything should work through intuitive hand gestures, without unnecessary menus or complicated controls.
I have a huge number of ideas for Dryad, and I'd love to bring them all to life. Now I just need more time!
