Inspiration

Most immersive media takes you somewhere else. We wanted the opposite: you stay on your own couch, and the story you read, hear or watch leaves small, believable traces in your real room. Only about 2–5% of what you see ever changes. The rest stays still, so the room feels like it is listening.

What it does

Vista is a hands-only media app for Meta Quest. Open it and your library appears in your room. Read a book on a real-feeling page, or press play on an audiobook and the app folds into a small player with word-timed captions.

As the story moves, the room answers it, at the moment the words happen:

  • an oak's roots creep from your real table's legs, and moss gathers on its edge;
  • a sunset lays water light across the table;
  • a tin of letters puts a letter beside you;
  • a lit lantern pools warm light;
  • a storm dims the room until only the lantern glows.

Nothing appears before you reach it, rereading rewinds it, and the room remembers your place.

How we built it

  • IWSDK 1.1 (WebXR), three.js, TypeScript, hosted as a URL, with zero runtime API calls.
  • Room understanding from planes, meshes and depth: it finds your table, its legs, the window and your feet, with fallbacks for unscanned rooms.
  • An offline story compiler turns text or a transcript into a spoiler-safe timeline of "trace intents". The same timeline follows a page or an audio playhead, and next, a film's timecode.
  • A trace engine fits intents to your furniture and grows each trace live in a worker. A change-blindness scheduler saves sudden changes for page turns or look-aways, and a budget keeps change under 5% of your view.
  • Grounding: depth occlusion, contact shadows and passthrough-matched colour.
  • Our own algorithms: space colonization (roots), falling-plate physics (leaves), Knuth–Plass line breaking (pages) and SLIC superpixels (0.926 boundary recall on BSDS500).
  • A day/night glass UI, CC0 materials, synthesized sound, and 1,000+ automated tests.

Challenges we ran into

  • Making a 2–5% change noticeable without breaking the calm.
  • Quest gives a table as a box with no legs, so we added a quick hands-only calibration.
  • Traces sharper than passthrough look fake, so we soften them to match.
  • Keeping text legible in passthrough.

Accomplishments that we're proud of

Roots that really seem to grow from your own table, one engine for reading and listening, and no controller anywhere.

What we learned

Restraint is the effect, grounding beats detail, and the best moment to change something is when no one is looking.

What's next for Vista – Living Media

Film and video on the same engine, any public-domain book or audiobook, on-device light matching, and shared reading across two rooms.

Built With

Share this project:

Updates

Submission history