Inspiration
I've always enjoyed creative art forms that work within constraints. For example, I love pixel art, where a limited resolution and colour palette can still allow for a huge amount of creative expression.
I wanted to explore a similar idea in mixed reality, how much creative freedom can a creator have given only a few basic hand interactions?
Hand tracking allows objects to be placed at any position, orientation and height, which makes it particularly suited to arranging things in 3D space. This led me to Japanese hakoniwa (box gardens), where small landscapes are created by arranging sand, rocks, trees and other natural materials.
What it does
Hakoniwa Studio lets you create miniature Japanese gardens using just your hands.
You can pick up a bowl of soil, and tilt it to pour the contents into your garden. You can also freely arrange rocks and trees, including partially burying them. To remove materials you've poured, you can pick up a small vacuum tool and use it to clear areas of the garden.
My goal was to make the creative process itself satisfying, while keeping the number of tools and interactions as small as possible.
How I built it
I used Godot with public domain photogrammetry textures for the art assets.
The garden is represented internally as a 64×64 grid of layered heightfields. Each cell holds a stack of materials, allowing the different substances to be poured on top of each other. A shader renders these layers as smooth spline surfaces.
Challenges I ran into and what I learned
Getting the pinch interaction and pouring materials to feel right was the first major challenge. Once those were working well, I knew there was something satisfying about the experience worth developing further.
A big challenge was figuring out how to let users edit what they had created. I spent a lot of time prototyping different approaches. I tried undo and redo buttons, directly manipulating sand with your hands, and using a bamboo ladle to scoop materials out of the garden. Without tactile feedback, they were not very easy and felt confusing.
Eventually I settled on a vacuum tool. It worked well because it reused the same pinch interaction users already knew, and its purpose was immediately understandable.
I learned that tactile feedback can play a big role in gesture controls, and finding a small set of interactions that feel good and work well together can be more valuable than offering lots of different tools.
Accomplishments that I'm proud of
I'm quite happy with the amount of creative expressivity achieved with only a small set of interactions and tools. When pouring and vacuuming came together, I was suddenly able to build gardens comfortably for fairly long stretches of time, which is a major achievement when making a creative tool.
What's next for Hakoniwa Studio
I'd like to further increase the creative expressivity of the existing materials. Some ideas I've had from real life hakoniwa include pruning trees, shaping rocks and flattening sand.
I experimented with these features during the competition, but finding interactions that were comfortable, intuitive and performant proved challenging, so I decided to leave them out for now.
I'd also like to add the ability to save and share gardens, so people can keep their creations and contemplate on other people's gardens.


