Inspiration
DJ and clubbing worlds in VRChat, DJing IRL, and artists like deadmau5, Skrillex and Fred again.. who turn a set into a show.
What it does
Immersive DJ Multiplayer is peer-to-peer multiplayer DJing with full-body-tracked avatars.
Note – we've added a link to the browser (non-immersive) version in the "Try it out" links for anyone wanting to try cross-platform multiplayer.
- Cross-platform – join in immersive mode from a Meta Quest or VR glasses headset, or from most browsers
- AI-powered styling – create visualisers and styling with AI via an MCP connector
- Async multiplayer – record your performance and share it
How we built it
- Core: Immersive Web SDK (IWSDK), React
- Networking & state: Ably, Upstash
- AI & tooling: Claude CLI, Muse Wonder, Muse Spark
- Build & deploy: Meta VR CLI, Vercel + Vercel CLI, GitHub + GH CLI
- Music: Suno for demo tracks, SoundCloud for the streaming music library
Challenges we ran into
Performance. Holding a steady frame rate with the IWSDK was the main challenge. We moved work onto web workers wherever possible and iterated with the Meta VR CLI performance monitoring tools. Several parts of the UI were customised beyond the IWSDK:
- Ray casting is only enabled for the active popup panel
- The track list is rendered to a texture, as too many UIKitML elements slowed performance
- Avatars are simplified after download to reduce draw calls per avatar
UX. The app has many popup windows, so we added a window layout manager to handle collisions, plus the ability to pick up, drop and orient popups using direct interaction.
Hand tracking. Custom pinch detection gives a faster response when DJing and grabbing faders, with grab hysteresis/lock-on so a grab isn't lost when the pinch state drops during rapid hand movement.
Accomplishments that we're proud of
- It all works! 6-player DJ sessions with VOIP and simplified IK, and the decks/host can be passed between players
- Mixed multiplayer between immersive players in XR and players in a browser
- MCP connector so anyone can vibe-code an audio visualiser or add stickers to the decks in near real time
- Fun features are easy to add, such as:
- Ball visualiser – a physics-driven ball the mix plays through, spatialised, so the sound changes as it flies around the room
- Mix follows room – seamlessly auto-mixes to a different track as you walk between rooms
What we learned
- AI loop-driven development, where the AI runs tests in the emulator before any manual testing, is now a great workflow
- IWSDK/web-based apps are incredibly fast to iterate on and are getting closer to game-engine performance (more improvements needed – WebGPU is hopefully on the horizon)
What's next for Immersive DJ Multiplayer
- More UGC – more customisation via MCP
- Avatar generation pipeline using Meshy or similar
- Auto DJ – a bot that DJs for you in your room, given a genre or playlist
- WASM audio engine for a performance boost
Built With
- android
- claude
- elevenlabs
- gemini
- iwsdk
- muse
- opensourceavatars
- peerjs
- pwa
- react
- soundcloud
- typescript
- upstash
- vercel
- whisper
- wonder