Inspo
HikuriXR began as an attempt to reproduce neurophenomenological experiences, the visual states reported in psychedelic and contemplative practice that resist measurement, in the room a person is actually standing in. I led the AI and XR initiative at Sensoria's Consciousness Conclave neurophenomenology study, and chose the effects from first-person reports that corroborated one another, art that people who had seen these states confirmed, and research on the visual cortex. In every source the patterns lie on real surfaces, follow the outlines of objects and develop from textures already in view. Mixed reality makes that possible: the stimulus is the live passthrough image, and the wearer stays in physical space.
Info
- Trip renders the form constants of geometric hallucination (tunnels, spirals, lattices, cobwebs) through the log-polar map from retina to visual cortex, seeded from edges in the camera image and placed at the depth of the surfaces that produced them. One dose control moves the view from patterns lying on the room to forms rising into view and a kaleidoscope.
- People in view are segmented and redrawn as lightning, anatomy, wireframes, light bodies with halos and wings, ASCII characters, or an invisibility cloak filled from remembered background.
- Spice, a network trained for the app, lets faces and creatures emerge from textures in the manner of DeepDream.
- Raymarched Mandelbulb, Menger and iterated-function-system fractals open as portals in the room; ferrofluid pools and field lines respond to the hands.
- A music studio with tracks generated on the headset from a spoken prompt drives every visual layer through its frequency bands, key, beat and drops. Tables become drum pads.
- Interaction is bare-hand only: direct touch, pinch and ray-pinch on 3D controls, and menus on the palm and wrist. Colocated headsets share one room and one music clock.
Creative Process
HikuriXR runs its whole AI stack on a Quest 3 and a paired phone, with no cloud. It is built on Unity 6 with URP on Vulkan, OpenXR with Meta's passthrough camera, depth and scene APIs, and XR Hands.
A C++ runtime with LiteRT, ONNX Runtime and whisper.cpp runs Stable Audio 3 small, YOLO26n-seg and Whisper on the headset. Stable Audio 3's diffusion transformer generates a 30 s track on the headset in 21.7 s while worn. Up to 9 networks share a session, including YOLO26 segmentation and pose, face recognition and Whisper. Live people masks blur bystanders or make them invisible, filling each body with the room behind it from memory. Spice distills DeepDream into one fast network; the phone runs true DeepDream by rewriting backpropagation as a forward graph. A trained neural cellular automaton grows texture across real walls and regrows it on each beat, beside pinch-seeded 3D creature swarms. Spice and Trip's cellular automaton were trained on cloud GPUs. The Quest's NPU is closed to apps, so a paired Galaxy S26 Ultra serves as its NPU over the local link and runs segmentation about four times faster than the headset alone. QNN profiling put 90% of YOLO's NPU time in its detection selection; moving it to the CPU cut 6.1 ms to 1.0 ms. Shaders reproject one camera's model output into both eyes at measured depth, so effects stay locked to the room's surfaces between model runs.
Challenges
Running a diffusion model beside passthrough rendering, and holding frame rate with several full-view layers, required staged model loading, low-priority inference threads and a render scale that follows the layer count.
Next
Face landmarks for Trip's mask, segmentation from a pinch, open-vocabulary segmentation and a Gaussian-splat capture of the room.
Target launch date
6 January 2027
Testing instructions
HikuriXR runs on Meta Quest 3 in passthrough with bare hands; controllers are not used. Sit with a table or wall within reach.
- Open HikuriXR and allow the permissions it asks for. The panels stand in front of you.
- Press the centre of the dial on the main panel with a fingertip to download the music model (1.27 GB, Wi-Fi needed, once). It reads "Models ready" when done.
- Press the centre again to generate the selected preset: a 30-second clip in about 22 seconds. The black orb to the upper left moves with the music.
- Turn your left palm toward your eyes and tap the glowing stud at the heel of the palm with your right index finger. The palm menu opens as a column of keys: Vision, Fields, Music, Room, System.
- Vision: choose Deep to label people and objects, a People look such as Lightning or Anatomy, a Room look such as Tiles, then Trip, and raise its dose. With Trip on and no People look, people appear as light beings with halos and wings.
- Rub your palms together for about 2 seconds: field lines and sparks run between them. Slam a palm on the table and a shockwave runs across it.
- Lay one hand flat on the table and hold it still. Press the round key that rises from the back of that hand with your other index finger and hold it, press one of the clip keys that stand beside the hand, and let go. The table now plays as drum pads.
- Tap the stud on the back of your left wrist for play, pause and skip.
Space Setup gives the room's walls and furniture; run it in the room you test in for the full effect. The phone link and shared rooms need extra hardware and are shown in the video.
