Deep Confusion

Project Story — deep confusion (final)

Inspiration

In quantum mechanics, a system holds every possibility at once — until you observe it. Measurement collapses the wavefunction: one outcome becomes real, and every other branch dies in front of you.

We realized this is also the physics of a human life. The road not taken keeps glowing precisely because you never measured it. The white rose you never picked stays luminous forever; the one you hold fades in your hand. Presence destroys possibility.

And it is the physics of this moment in AI. Generative models are literal superposition machines — a prompt is a wavefunction, sampling is a measurement, and every image you accept is a million siblings you killed. Meanwhile the attention economy is industrial-scale anti-collapse infrastructure: systems engineered to keep every decision half-open, every elsewhere glowing, so you never stop scrolling.

We wanted to build a world where you could feel all of this in your body — not read about it.

What it does

deep confusion is a browser-based playable world where observation is the only verb.

You describe a place in natural language. An LLM composes it as a structured voxel world — raw, axis-aligned volumes floating in a void, each carrying its own physical nature (solid or passable, mass, friction). This is the world in superposition: unmeasured, undefined, glowing. Beneath the voxels lies a generated Gaussian-splat substrate — the ground of the scene, the world you actually believe in — tiled forward indefinitely, so the boundary that normally ends a generated world never arrives. As you walk and look, a real-time diffusion model "collapses" what's in your gaze into photorealistic imagery, streamed back into the browser. Denoise strength is mapped to observation intensity — the harder you look, the more definite (and the more dead-final) the world becomes. And what collapses stays collapsed. The LLM acts as the world's hippocampus, committing every observation to a persistent record and feeding it back as conditioning. Observation is not just rendering — it is written into memory, which is exactly what makes collapse irreversible. Here is the twist you feel in your bones: only the abstract geometry is solid. The physics engine collides you with the raw uncollapsed volumes. The beautiful, realistic, collapsed appearance has no physical presence at all — you pass straight through it, like light through a ghost. And one thing in this world is not generated: a real human, volumetrically captured and reconstructed in 4D, standing among the voxels. The world is generated; the person in it is not.

The abstract is what holds you up. The realistic is untouchable. What you can grasp is never what you see; what you see is never what you can grasp.

How we built it

The pipeline is a deliberate collision of six systems, all free and all runnable against a browser — with the language model playing two roles: world-builder and hippocampus.

World Labs → 3DGS substrate. A generated Gaussian-splat environment provides the ground truth of the scene — the world you actually believe in. We tile it forward indefinitely, so the boundary that normally ends a generated world never arrives. LLM → voxel world. A language model outputs a structured voxel layout — position, scale, label, and physical properties (solid or passable, mass, friction) — in the spirit of the LayoutGPT / Holodeck family, but we deliberately stop before asset retrieval: the raw volumes are the point, and the LLM's semantic judgment directly becomes collision physics. Spark → 4DGS inhabitant. Our own volumetrically captured human, reconstructed through our 4DGS pipeline, is rendered by a modified Spark into the same three.js scene graph as the voxels — splats and raster geometry sharing one depth-correct pass. The world is generated; the person in it is not. Three.js renders the raw geometry in the browser at 60fps. StreamDiffusion performs real-time img2img on the rendered frames — the camera feed goes out over WebSocket, comes back stylized, and is composited as the final visual layer. Denoise strength is driven by gaze dwell time. LLM as memory. The language model also acts as the world's hippocampus: it keeps a persistent record of what the diffusion model has generated in each region and feeds it back as conditioning, so the video model stays consistent with its own past. Observation is not just rendering — it is committed to the record, and the record persists. Rapier (@dimforge/rapier3d-compat) runs collision entirely client-side, bound only to the LLM's voxel volumes.

The critical architectural decision: physics and diffusion are fully decoupled. Movement and collision run locally at 60fps with zero dependence on the AI layer; the diffusion stream is asynchronous and latency-tolerant. The world stays playable even when the imagery lags — which turned out to be conceptually perfect: reality arriving a beat late, the way meaning always does.

Hard constraint we held the whole way: no paid APIs. Everything runs in the browser plus a local GPU.

Challenges we ran into

Latency vs. feel. Our first architecture routed player movement through the diffusion loop — unplayable. Decoupling physics from rendering wasn't just an optimization; it forced us to discover the piece's central mechanic (solid abstraction, ghostly realism). Two kinds of geometry, one depth buffer. Gaussian splats and raster voxels do not naturally coexist. Getting the 4DGS human and the semantic volumes into a single depth-correct pass — so a generated box can genuinely stand in front of, or behind, a real person — took a modified Spark renderer and most of a night. Making a constraint mean something. Real-time diffusion at hackathon scale can't texture a whole world — so we made partial collapse the aesthetic. The limitation isn't hidden; it's the artwork. Temporal coherence. Frame-to-frame flicker in the diffusion stream initially felt like a bug, until we recognized it as the shimmer of a state that hasn't fully decohered — and tuned it instead of suppressing it.

What we learned

The best architectural choices are the ones where the technical limitation is the concept. We stopped fighting our constraints — latency, flicker, untexturable geometry — and let each one carry the metaphor. Build the playable skeleton first; let the AI layer arrive late, like observation always does.

What's next

This prototype is the first chamber of a larger trilogy, deepconfusion.ai, about the three irreversible collapses of the machine age — creating, loving, and remaining oneself. Next: a chamber on deep distraction — the interference pattern that keeps pulling even after you've chosen — and an exit door designed to be passed through many times, because no one collapses only once.

Built With

• 3dgs
• 4dgs
• cuda
• gaussian-splatting
• javascript
• llm
• node.js
• particle
• python
• pytorch
• rapier
• stable-diffusion
• streamdiffusion
• three.js
• typescript
• vite
• webgl
• websockets
• world-labs
• worldlabs