Inspiration
What it does## Inspiration
Spatial computing in mixed reality has largely replicated flat 2D monitors floating statically in a void. We asked a fundamental question: What if the operating system itself was an intelligent, autonomous spatial agent that arranges, prioritizes, and reacts to your focus in true 3D space?
With Meta Quest 3's high-resolution color passthrough, inside-out 6DoF SLAM tracking, and the Snapdragon XR2 Gen 2 platform, we engineered MetaSpatial OS—an agentic spatial operating environment that doesn't merely display tools; it actively collaborates with the user.
What It Does
- Autonomous Spatial Architecture: Automatically arranges floating 3D windows in ergonomic cylindrical or spherical arcs dynamically calculated around the user's natural gaze and field-of-view (FOV 110°).
- Interactive Multi-Agent Hologram: A central 3D reactive avatar acts as the Llama 3 copilot, pulsing with real-time telemetry, orbiting audio visualizer bars, and task execution states.
- 3D Knowledge & Task Graph: Visualizes agent task execution as interconnected spatial nodes, allowing the user to click, inspect, and trigger sub-tasks.
- WebXR 6DoF & Zero-Install Fallback: Runs natively in Meta Quest Browser in full immersive VR with hand/controller raycasting, while simultaneously supporting desktop and mobile browsers via mouse-based orbit and ray hit-testing.
- Bilingual English/Arabic Interface: Instant localized audio telemetry, speech synthesis, and typography for global and MENA accessibility.
- Spatial Audio & Procedural Sound: Generates directional audio cues and voice synthesis directly from browser APIs without external asset latency.
How We Built It
- Frontend & 3D Rendering: Three.js (r128 offline bundle), WebXR Device API, custom native Orbit & Touch controller, CanvasTexture dynamic UI baking.
- Agent Backend Engine: Python 3.10+, featuring custom 3D linear algebra (Vector3 dot/cross, Quaternions, Kay-Kajiya Ray-AABB intersection testing) and a tri-agent pipeline (SpatialArchitectAgent, TaskReasoningAgent, SpatialToolExecutor).
- Cloud Infrastructure & Production Hosting: High-performance Hostinger edge deployment with HTTP/3, TLS 1.3, and global CDN caching at https://meta.elnemer-ai.com.
- Audio & Speech: Web Audio API (procedural oscillator synthesis for zero-latency laser/pulse feedback) and Web Speech API (SpeechSynthesis).
Challenges We Overcame
- Ergonomic Spatial Layouts: Calculating window coordinates that fit within Quest 3's 110° FOV without causing neck strain required implementing cylindrical projections with alternating vertical staggering.
- Raycasting Precision in 3D: Ensuring pointer rays accurately hit floating window frames across varying angles required writing and mathematically validating an AABB slab intersection algorithm with strict $< 10^{-6}$ error bounds.
- Zero External Dependencies: Eliminating fragile external CDN scripts by packaging Three.js and building inline camera controllers to guarantee 100% offline uptime and instant load times.
Accomplishments That We're Proud Of
- Rock-solid 90 FPS rendering in Meta Quest 3 with zero frame drops.
- Production deployment live on public web and accessible globally at https://meta.elnemer-ai.com.
- 100% verified mathematical precision on all 3D vector transforms and raycast collisions.
- Bilingual Arabic/English support expanding spatial computing to the Arab world.
What We Learned
- Spatial computing demands agentic intelligence to manage multi-window clutter—dynamic auto-arranging is essential for long productivity sessions in VR.
- Procedural audio synthesis delivers instant, zero-latency spatial feedback far superior to fetching static sound files over network links.
What's Next for MetaSpatial OS
- Direct integration with Meta Horizon OS Native APIs and Shared Spatial Anchors for multi-user collaboration.
- Local on-device Llama 3 8B inference running on Quest 3 via WebLLM / ONNX Runtime Web.
- Hand-tracking pinch-to-drag repositioning of individual floating workspace panes in MR passthrough.
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for MetaSpatial OS: Autonomous Agentic Workspace
Built With
- javascript
- llama-3
- meta-horizon-os
- meta-quest-3
- python
- three.js
- web-audio-api
- webxr