The Flow - Resonance with the world

anti-dote for brain-rot

Comment

The Flow

Resonance with the World

The Flow is a gesture-driven XR journey inspired by Daoist philosophy, Tai Chi, and Chinese ink-wash art. Instead of controlling or conquering a virtual world, the player moves in harmony with it—awakening landscapes, transitioning between realms, and eventually returning everything to 道, the Dao.

Inspiration

Most digital experiences ask us to move faster.

Scroll faster. React faster. Complete the task. Control the environment.

The Flow explores the opposite relationship.

Inspired by Daoist philosophy, we asked:

What if progress came not from force, but from harmony?

In The Flow, the player is not entering a traditional game. They are witnessing the birth and transformation of a world.

They do not create it. They do not command it. They become synchronized with its emergence.

Our story was inspired by the Daoist idea that all things arise from the same source, take countless forms, and eventually return to one.

We combined this philosophy with Tai Chi-inspired movement, traditional Chinese ink aesthetics, immersive sound, and Gaussian-splat environments to create an experience that feels contemplative rather than competitive.

The journey begins in emptiness, awakens the earth, moves through bamboo and mountains, expands into the cosmos, and concludes when every world dissolves and reforms as the Chinese character:

道

What it does

The Flow is a controller-free XR experience where the player uses natural hand movements to reveal and transform immersive worlds.

The Void

The experience begins in a completely white space.

There is no ground. No sky. No interface. Only silence and a faint heartbeat.

A glowing circle appears in front of the player.

A voice says:

"Before all things, there was only Dao."

The Awakening of Earth

When the player steps into the circle, the ground begins to expand beneath them.

A mesh grows outward into the empty space. Sound slowly enters the world. The first wind can be heard.

The ground is not simply generated. The earth awakens.

The Bamboo Forest

The player is invited to raise both hands.

As their hands rise, the white world opens and reveals a living Gaussian-splat bamboo forest.

Bamboo surrounds the player. Mist floats between the trees. Light filters through the leaves.

The world does not appear because the player cast a spell. It is revealed because the player has aligned with its movement.

The Mountain

A new interaction appears: a slow, continuous circular gesture inspired by the flow of Tai Chi.

As the player completes the loop, the bamboo forest begins to dissolve.

Ink-like particles and splats move through the space. The forest transforms into a mountain summit above the clouds.

The player now stands between earth and heaven.

The Cosmos

The player continues the same circular gesture.

The mountain breaks apart into light, mist, and splat fragments. The horizon disappears. Stars emerge.

The player enters a cosmic environment filled with distant celestial forms and a planet suspended far away.

The same movement that guided leaves and landscapes now reaches across space.

Resonance

By continuing the circular gesture, the player influences the distant planet.

The planet rotates, shifts, and responds to the rhythm of the player's movement.

It is not treated as an object being dragged or controlled. Instead, the player and the planet appear to move in resonance with each other.

The interaction grows from a personal gesture into a connection with the larger universe.

Return to Dao

Eventually, the gesture ends.

The planet slows. The stars begin to disappear. The cosmos dissolves.

Fragments of the mountain, bamboo forest, earth, and sky return as ink, particles, and splats.

Everything gathers in front of the player and forms one living Chinese character:

道

All the worlds were expressions of the same flow. Everything returns to its source.

How we built it

We developed The Flow as a browser-based WebXR experience, allowing the player to enter the journey through an immersive headset without installing a traditional game application.

The primary experience combines:

  • WebXR
  • Hand tracking and gesture recognition
  • Browser-based real-time 3D rendering
  • Gaussian-splat environments
  • World Labs Marble
  • Animated meshes
  • Particle and splat transition effects
  • Spatial sound
  • Gesture-driven scene sequencing
  • Interactive celestial objects
  • Chinese ink-wash-inspired visual design

The player's hands serve as the primary interface.

Instead of displaying traditional buttons or menus, the experience detects larger body movements that support the story.

The first major interaction recognizes when the player raises both hands. This movement triggers the reveal of the bamboo forest.

The second interaction tracks a continuous circular gesture. We analyze the movement of the player's hand over time, including its direction, continuity, radius, and completion.

This gesture is reused throughout the journey:

  • To transition from the bamboo forest to the mountain
  • To transition from the mountain to the cosmos
  • To influence the distant planet
  • To establish a consistent movement language across the experience

We used World Labs Marble to create detailed spatial environments that could support the bamboo, mountain, and world-transition concepts.

Gaussian splats allowed us to preserve the organic visual complexity of these environments. We combined them with traditional meshes, particles, lighting, and sound so the worlds could become interactive rather than remaining static scenes.

PICO interaction prototype

Alongside the primary WebXR experience, we developed an experimental Unity prototype for PICO 4 Ultra.

This prototype was created to test:

  • Controller-free interaction
  • PICO high-frequency hand tracking
  • Tai Chi-inspired circular gesture recognition
  • Gesture speed and continuity
  • Player feedback without physical controllers
  • Standalone Gaussian-splat performance
  • Adaptive spatial guidance

The PICO prototype uses a simplified river-crossing experience in which successful movements raise stepping stones.

This was not designed as a replacement for the main story. It functioned as a focused test environment where we could evaluate whether players understood the gestures, whether the tracking remained stable, and how quickly feedback should appear.

The PICO prototype was built with:

  • Unity 6
  • C#
  • PICO Unity OpenXR SDK
  • Unity OpenXR
  • Unity XR Hands
  • PICO high-frequency hand tracking
  • ARM64
  • Vulkan
  • A runtime Gaussian-splat renderer
  • A local interaction and progression system
  • Unity EditMode gesture tests

The prototype's adaptive guidance layer evaluates the player's movement speed, continuity, hand position, and completion state. Based on those signals, it changes the audiovisual feedback and progression state.

This helped us explore how a spatial system could guide a player through a natural movement without relying on menus, controller vibrations, or constant spoken instructions.

Challenges we ran into

Designing reliable gestures

Natural hand movement is expressive, but it is also inconsistent.

Different players make circles at different speeds, sizes, angles, and heights. Hand-tracking data can also contain noise, temporary loss of tracking, or sudden position changes.

We needed the gesture system to recognize intentional movement without requiring the player to behave like a perfect input device.

We worked on balancing:

  • Recognition accuracy
  • Movement flexibility
  • Gesture completion time
  • False positive prevention
  • Tracking interruptions
  • Player comfort

Communicating without traditional UI

The Flow is designed to avoid conventional buttons and menus.

This created a communication challenge: how does the player know what to do?

We explored using spatial placement, particles, hand-responsive visuals, environmental movement, narration, and sound effects to communicate:

  • When the player's hands are detected
  • When the gesture has started
  • Whether the gesture is being performed correctly
  • How close the player is to completing the movement
  • When the world is ready to transform

Rendering Gaussian splats in XR

Gaussian-splat worlds can contain rich visual detail, but XR platforms have strict performance requirements.

We had to consider:

  • Splat size and density
  • Standalone headset limitations
  • Loading time
  • Memory usage
  • Frame rate
  • Visual quality
  • Transition performance

The standalone PICO prototype was especially useful for testing how a splat environment behaves under mobile XR hardware constraints.

Creating meaningful transitions

A basic scene cut would have weakened the story.

The bamboo forest, mountain, cosmos, and final 道 symbol needed to feel like different expressions of the same world.

We experimented with splat dissolves, particles, opacity changes, mesh movement, environmental audio, and lighting transitions to make each realm flow into the next.

Maintaining one clear project identity

Because we explored WebXR, PICO, adaptive guidance, splats, and generated worlds, it was easy for the project to become too broad.

One of our most important design challenges was deciding what served the core experience.

We returned repeatedly to one principle:

The player does not control the world. The player resonates with it.

Accomplishments that we're proud of

We are proud that The Flow connects its philosophy directly to its interaction design.

The player cannot complete the experience by rapidly pressing buttons or forcing the environment to respond. They must slow down, observe the rhythm of the world, and complete deliberate movements.

We are especially proud of:

  • Creating a controller-free XR interaction system
  • Using Tai Chi-inspired movement as the central interface
  • Building a consistent circular gesture across multiple scenes
  • Connecting hand movement to environmental transformation
  • Creating immersive environments with World Labs Marble
  • Combining Gaussian splats with real-time XR interaction
  • Testing standalone splat rendering on PICO 4 Ultra
  • Building both a WebXR experience and a focused Unity interaction prototype
  • Designing transitions from bamboo, to mountain, to cosmos
  • Making the final 道 formation part of both the visual climax and the philosophical meaning

Most importantly, we are proud that the technology supports the emotional purpose of the project.

The hand tracking, splats, sound, and environmental effects are not separate demonstrations. They work together to create a single experience about harmony.

What we learned

We learned that gesture design is also a form of choreography.

A gesture is not defined only by its coordinates. Its speed, scale, rhythm, direction, and visual response determine how it feels.

The same circular movement can feel like:

  • A command
  • A spell
  • A physical exercise
  • A meditative practice
  • A conversation with the environment

We learned that players need feedback throughout a gesture, not only after completing it. Subtle visual and audio responses help the movement feel connected to the world.

We also learned that Gaussian-splat generation is only one part of building an immersive experience.

A generated world still needs to be:

  • Optimized
  • Loaded
  • Rendered
  • Positioned
  • Sequenced
  • Made interactive
  • Connected to the narrative

The PICO prototype taught us the importance of testing interactions in a simplified environment before integrating them into the complete experience.

Finally, we learned that emerging technologies become more meaningful when they share one creative purpose.

World generation, hand tracking, spatial computing, sound, and adaptive systems should not compete for attention. They should disappear into the experience.

What's next for The Flow

Our next goal is to make the transitions between worlds more continuous and physically expressive.

Rather than fading between separate environments, we want:

  • Bamboo stalks to stretch and reform into mountain shapes
  • Leaves to become clouds
  • Rocks to dissolve into stars
  • Water droplets to become planets
  • Fragments from every world to form 道

We also plan to improve the gesture-recognition system so it can adapt to different players, movement ranges, body positions, and speeds.

Future development includes:

  • More responsive hand-tracking feedback
  • Adaptive gesture thresholds
  • Improved standalone XR performance
  • More interactive Gaussian-splat environments
  • Spatial audio that evolves across the journey
  • Environmental reactions to movement speed and proximity
  • Additional ink and splat transformation effects
  • A more expressive cosmic planet interaction
  • Optional voice-guided accessibility
  • Personalized world generation through World Labs
  • A longer journey based on additional Daoist concepts

Our long-term vision is to create a contemplative spatial experience where natural movement becomes a way of communicating with virtual worlds.

Not through control. Not through force. Through resonance.

Tracks

Storytelling & Entertainment Project

The Flow tells its story through embodiment.

The narrative is not delivered only through dialogue or cinematics. The player physically participates in the awakening and transformation of the world.

The progression from white void, to earth, bamboo, mountain, cosmos, and finally 道 represents a journey from emptiness into form and back to the source.

Interaction, environment, sound, pacing, and visual transformation all communicate the same idea:

Harmony cannot be forced.

Interactive World Experience powered by World Labs

We used World Labs Marble to create immersive spatial environments that form the living worlds within The Flow.

These environments are not presented only as passive backgrounds. They are revealed, transformed, and connected through the player's physical movement.

Our goal is to explore how generated Gaussian-splat worlds can become active parts of an interactive narrative.

The player does not simply view a bamboo forest or mountain.

They awaken it. Move through it. Transform it. And eventually watch it return to Dao.

Agentic Spatial System powered by PICO

Our experimental PICO 4 Ultra prototype explores how an adaptive spatial guide can support natural, controller-free interaction.

The system evaluates real-time hand-tracking information, including:

  • Movement speed
  • Gesture continuity
  • Hand position
  • Gesture progress
  • Completion state

It then selects appropriate visual, audio, and progression feedback to help the player maintain the intended movement.

Rather than relying on buttons or controller input, the spatial system interprets the player's physical behavior and adapts the experience around it.

The current prototype focuses on gesture guidance and progression. Future versions could expand this system with voice interaction, personalized pacing, player reflection, and dynamically selected World Labs environments.

Built With

  • android-xr
  • arm64
  • c#
  • gaussian-splatting
  • javascript
  • particle
  • pico-4-ultra
  • pico-hand-tracking
  • pico-unity-openxr-sdk
  • typescript
  • unity-6
  • unity-openxr
  • unity-xr-hands
  • vite
  • vulkan
  • webxr
  • world-labs-marble
Share this project:

Updates