LA30B Paul, John, George, Ringo, and You!

Inspiration

We wanted to build something that could only exist in headset: not a flat game ported into VR, but an experience where the room itself is the instrument. The Beatles felt like the right story to tell that way — four eras, four looks, one band that keeps growing around you — and Gaussian splats felt like the right medium, since a splat scene is basically a photograph you can stand inside of. The pitch was simple: what if picking up an air guitar and playing well enough could physically reshape the world in front of you, era by era, until the whole band is on stage with you?

What it does

LA 30: Paul, John, George, and You is a WebXR air-guitar rhythm game for Apple Vision Pro. You start alone with one Beatle in a Gaussian-splat recreation of a real space — Ed Sullivan Show soundstage, the Ashram, Yellow Submarine, Abbey Road — and a beat map drives an approaching note highway. Strum with your right hand, fret with your left; hit your notes and a new Beatle joins, the world dissolves front-to-back into the next era, and the splats themselves react to how well you're playing: a bloom ripples out from a Perfect hit, the whole palette warms and saturates on a hot streak, a bad hit sends a shudder through the scene. By the end, all four Beatles are jamming together on the Abbey Road crossing.

Only tested on Apple Vision Pro!

Trick is to use just your wrist not your whole arm! If you are on desktop just press the J, K, or L, keys when the notes intersect with the bar!

How we built it

Three.js + the WebXR Device API for the render/session layer, and Spark (World Labs' Gaussian-splat renderer) for the environments. The core technical bet was forking Spark's splat shader — injecting custom GLSL via its dyno modifier system to expose uniforms for beat phase, FFT audio bands, and player streak, plus an 8-slot ring buffer of "waves" that each strum emits into the world. Every splat sums the active waves by distance and time-since-emit and blends toward one of a few procedural morph targets, so a strum doesn't just score points — it visibly ripples through the environment.

The rhythm layer runs off a single Web Audio conductor clock (no stems — we route the one music track through a low-pass filter + gain that opens up as you play well and muffles when you don't). Hand input comes from the WebXR Hand Input Module: vertical velocity spikes on the strum hand become timestamped strum events with amplitude and polarity; the fret hand's height maps to wave frequency, and a pinch latches a sustained "held" wave. Beatles run on a shared two-clip (idle/play) AnimationMixer setup, and an "Era Director" state machine ties it all together — watching score against per-section thresholds and triggering the world transition, the next Beatle, and the next song section together.

Challenges we ran into

The biggest one was almost invisible: our captured splat scenes came in at 1.9–3 million splats each, roughly 10–20x over what Vision Pro's stereo renderer can comfortably draw. That's what caused real stuttering and, worse, outright crashes on-device — we eventually traced it to a splat re-decimation step firing on VR session entry, which briefly spiked memory hard enough to crash Safari. Pulling that out and settling on one shared, pre-decimated splat budget for both desktop and headset fixed it.

We also hit a genuinely funny GLSL bug: we named a shader function parameter smooth, which is a reserved keyword in GLSL, and it silently broke shader compilation. And because every Beatle model came from a different pipeline (Mixamo rigs at very different native scales, one model that split into two textures we had to composite for a working face texture), something as simple as "put the guitar in his hands" turned into building a live in-browser tuning tool — sliders that move the guitar in real time on the actual rigged character so we could dial in position/rotation/scale per Beatle and copy out the final JSON.

Accomplishments that we're proud of

Owning the splat shader end-to-end — uniforms, wave physics, era-transition dissolve, all hand-written GLSL — instead of treating Spark as a black box. Getting a full playable loop working: four real transitions, four Beatles accumulating on stage, and a beat map that's actually tuned to the track's measured BPM. And getting it running acceptably in an actual Vision Pro headset, which forced real performance engineering (splat decimation, foveated rendering, framebuffer scaling, back-of-head splat culling) rather than just a desktop demo that happens to support WebXR.

What we learned

That "VR lag" almost always traces back to one dominant cost — for us it was splat count, not shader complexity or draw calls — and that fixing the actual bottleneck matters more than optimizing everywhere at once. That runtime decimation is a fine stopgap but a real memory risk on-device; the correct fix is decimating source assets offline. And that rigged-character work (bone-parenting props, texture compositing, cross-model scale differences) benefits enormously from building a small live-tuning tool instead of guessing coordinates blind — it turned a frustrating trial-and-error loop into something we could dial in in minutes.

What's next for LA 30: Paul, John, George, Ringo, and You!

Offline, importance-aware decimation of the splat scenes (via SuperSplat) to push visual quality up while keeping the VR-safe splat budget — right now we trade sharpness for headset stability at runtime. Beyond that: refining the hand-tracking recognizer with real on-device playtesting, more songs and eras, and eventually a licensing-cleared path to a public release, since right now this is an unlicensed fan concept built for the hackathon with original casting on the character likenesses and a placeholder track.

Built With

• blender
• claude
• tripo
• webxr
• worldlabs