Inspiration
Laptops and phones have plenty of tools for capturing and organizing ideas. VR has almost none, and the reasons are clear: headsets are niche, typing is slow, and wearing one for hours is uncomfortable. The few apps that do exist try to put a flat desktop inside a headset, which ignores what VR is actually good at: space, interactivity, and immersion. We wanted something closer to Notion or Obsidian, but built for a headset, where your ideas surround you instead of sitting in a sidebar. That became ZAP!.
What it does
ZAP! is a spatial knowledge space for Meta Quest. You step into an immersive world and turn ideas into nodes you can place, move, and connect around you.
A session starts with your voice. Hold a button on the controller and say a topic, and a glowing orb appears in your hand. An AI writes its note, using the ideas it descends from as context. You place the orb by throwing it, link it to related orbs by pointing from one to the other, and pull or stretch the whole map with two hands.
Every orb is also a doorway. Throw one forward and you travel into a 3D world of that topic that you can walk around in. Inside, you can grab and resize floating learning objects, ask questions out loud, and read the answers on glass panes that stay where you put them. You can even reshape the scene by voice: "create RNA and delete the DNA."
How we built it
We built ZAP! in Unity with C# and deployed it to Meta Quest using the Meta Quest developer platform.
- Worlds. We used World Labs' world models to generate the immersive environments. For the demo we generated three ahead of time (the Great Wall, the inside of one of its watchtowers, and a DNA biology lab) and exported them as textured 3D meshes that the headset loads at run time.
- Interface and objects. The interface was designed in Figma and brought into Unity through one shared theme. The learning objects inside each world are not hand-modeled: an AI describes them using a small set of simple shapes, and Unity builds them on the spot.
- Nodes. Each node stores its topic, its note, its position and its links in a small file on the headset. Everything you see is created from code when the app starts.
- Input. Input is controllers plus voice. A small Python service on a laptop transcribes speech locally with Faster Whisper and calls GPT-6 Luna to write notes, answer questions and edit objects.
Challenges we ran into
- Scope. Our biggest debate was how broad to go. A general "think anything" tool risked feeling empty, and a tool for one narrow use case risked feeling like a toy. We settled on a general tool shown through three concrete worlds, so the demo has depth without locking the product to one subject.
- World generation limits. Each World Labs world cost about 5,000 credits, so we could afford exactly three. We each had to make multiple accounts to create enough world assets.
- Video is not a place. We first tried wrapping live generated video around the user. It drifted and never held together as one scene, so we moved to worlds with real geometry you can walk on.
Accomplishments that we're proud of
We shipped a working end-to-end concept on a real headset that is a great foundation for a lot of use cases. Creating and connecting nodes in 3D already feels natural: the orb is in your hand the moment you press the button, and you never touch a keyboard. We also put real effort into the interface so it feels native to VR rather than a 2D app floating in space, and the codebase is a solid base to keep building on.
What we learned
Most of us were new to Unity, C#, 3D modeling and Quest development, so the learning curve was steep. The biggest lesson was that typing kills immersion, so input has to be spatial and voice-first. The second was that world models enable user-tailored immersive experiences generated at runtime—this is the crucial leverage that runs our app.
What's next for Zap!
We want to turn ZAP! into a real product. Logistically, this means providing even better immersive experiences at a lower cost. Though this is not feasible currently as world models are far too nascent still, we predict the XR industry will close this gap in the near future. Feature-wise, next on our list is generating brand-new worlds from a spoken topic while you wait, AI-suggested links between related ideas, and shared spaces for study groups. We're planning a launch on Product Hunt. Come follow along.
Built With
- quest
- unity
- worldlabs