-
-
A photo freezes a moment, a video replays it, a HoloMoment lets you step back inside
-
On Quest 3 the hologram stands on your real table
-
A video diffusion model imagines 48 cameras around her, even her back
-
One Gaussian splat model per frame: 121 frames, ~300,000 splats each
-
Three phone videos, three moments
-
The same app scans tables, rooms and monuments
-
In VR the moment stands inside a scanned living room
-
Runs in the browser, built on standard Gaussian splat files
Inspiration
Photos freeze a moment. Videos replay it. We wanted to step back into one. Volumetric capture, where you can walk around a moving person, still needs a studio ringed with synchronized cameras. We wanted it from a few seconds of phone video of someone you love.
What it does
HoloMoments turns a four-second phone video, filmed from one side only, into a 4D hologram: 121 frames of Gaussian splats you can see from every angle, even the back the phone never saw.
On Quest the moment lands on your real table in mixed reality, using the detected planes, or floats in front of you in VR. Everything works seated and with hands alone:
- pinch the hologram and circle your hand to turn it;
- pull with both hands to resize it;
- pinch the timeline to scrub frame by frame;
- tap the panel for slow motion, the next moment, life size or a photoreal scanned living room.
One button switches between mixed reality and VR. In mixed reality your real hands stay in front of the hologram.
The same phone app also scans tables, rooms and monuments into Gaussian splat scenes, and those scans become places: a moment can stand on the dining table of a scanned living room.
How we built it
- Our server is one PC with a 16 GB GPU. SAM 3D Body tracks the body pose in every frame.
- 4DAnyone, a multi-view video diffusion model, generates the person from 48 virtual cameras in two rings. It normally needs 22 GB of VRAM; we stream its 30 transformer blocks through two GPU slots and chunk the heavy layers.
- BiRefNet masks seed a point cloud for every frame, and gsplat trains one splat model per frame that follows the skeleton.
- Our converter bakes view-dependent colour into one colour per Gaussian, keeps the 100k Gaussians that matter most (98.6% of the image) and pins the feet to the plinth to remove drift from a hand-held phone.
- The Quest app is WebXR: three.js with World Labs' Spark renderer, WebXR hand input and plane detection.
Challenges
- Loading inside an immersive session froze the Quest, so everything now loads, uploads to the GPU and compiles before you enter.
- Pruning by opacity and size made the hologram glitter. Pruning by measured render weight fixed it.
- People look down at a miniature, where the generated views were weakest, so we added a second ring of cameras 40° above.
What's next
- Film to headset in one flow: your moment appears in your Quest library after processing.
- Spatial audio from the original clip.
- Shared viewing for two people around the same table.
- Temporal compression to make moments 5–10× smaller.
- Target launch: a free web app in Q1 2027, then a native Horizon Store release.
Try it
Open https://holo.lisavets.dev in the Quest Browser, wait until loading reaches 100%, tap Enter mixed reality and allow hand tracking and spatial data. On a computer, click "No headset? Try the Quest version" for the built-in Quest 3 emulator.
Built With
- 4danyone
- birefnet
- firebase
- gaussian-splatting
- gsplat
- javascript
- python
- pytorch
- sam-3d-body
- sparkjs
- three.js
- webxr
