AetherDesk: Seated Hands-First Spatial Agentic Workspace

Seated, hands-first spatial workspace turning complex thoughts into tactile data capsules guided by an embodied autonomous AI agent.

Comment

💡 Inspiration

Spatial computing promised infinite monitors, but reality gave us "Gorilla Arm" shoulder fatigue in under 5 minutes. Reaching for floating 2D browser windows 1.5 meters away directly violates human biomechanics.

I asked: What if spatial productivity respected the human body? What if your entire workspace lived in an ergonomic 24-inch tactile micro-sphere above your lap, completely controller-free, operating effortlessly whether seated in an office chair, a wheelchair, or a coach airplane seat during a commute?

🥽 What AetherDesk Does

AetherDesk is an ergonomics-first, hands-only spatial workspace built for Meta Quest 3, Quest 3S, and Meta VR Glasses:

  • 24-Inch Seated Lap Envelope (The Airplane Seat Test): Constrains all interactive cards, slots, and tools within a 0.6 m radius above the lap. Elbows stay naturally anchored on armrests.
  • Tangible 3D Data Capsules: Ideas, tasks, and notes materialize as tangible 3D cards rendered in real-time with dynamic CanvasTextures, priority tagging, and magnetic slot snapping.
  • Embodied Autonomous Co-Pilot ("Aether"): A 3D gyroscopic entity situated on the lap console pedestal that perceives proximity, tracks held cards, and autonomously executes spatial tool calls (auto-aligning slots, decomposing complex goals into subtasks, and synthesizing cross-card executive rollups).
  • Daily Retention Rituals (Best Reason to Come Back): Pre-configured daily productivity routines:
    • 🌅 Morning Triage: 5-minute cold start for commute planning into Slots 0 & 1.
    • ⚡ Deep Focus Sprint: Dims ambient scene lighting and isolates active focus cards.
    • 🌙 Evening Review: Auto-synthesizes an Executive Brief and increments the persistent daily streak (🔥 Day Streak).
  • 1-Click 60-Second Guided Tour (Best First Five Minutes): An automated choreography demonstrating micro-pinch grab, wrist palm palette, co-pilot goal decomposition, and dock alignment in under 60 seconds.

🛠️ How I Built It

Built 100% client-side from scratch for Meta VR Start 2026:

  • WebXR Hand Tracking API: Full 25-joint articulated cybernetic hand skeleton, 2.4 cm micro-pinch detection, and palm-up wrist palette summoning.
  • Three.js (r186): Custom curved console geometry, ACESFilmic tone mapping, and dynamic CanvasTexture serialization.
  • Procedural Web Audio Engine: Synthesizes mechanical clicks, magnetic snaps, wooshes, and harmonic chimes client-side without external audio files.
  • 100% Free & Offline-First (The Take-It-Away Test): Zero paid third-party APIs. Operates completely standalone via local spatial NLP rules, Web Speech API, and persistent IndexedDB/LocalStorage vault.

🧗 Challenges I Ran Into

  1. Avoiding WebXR Session Freezes: Standard browser dialogs (window.prompt) freeze immersive VR on the Meta Quest Browser. We designed a custom in-app non-blocking Spatial Thought Modal with quick-suggestion chips.
  2. Hand Skeleton Performance on Mobile Silicon: Tracking and rendering 25 joints per hand across 60–90 FPS required batching bone lines into dynamic THREE.LineSegments to minimize draw calls.
  3. Ergonomic Envelope Tuning: Seated postures vary widely. We engineered a real-time calibration system allowing users to adjust console elevation ($\pm 12\text{ cm}$) and reach ($\pm 15\text{ cm}$) for wheelchair and recliner accessibility.

🏆 Accomplishments I am Proud Of

  • Passing all Four Meta Heuristic Tests (Airplane Seat, One Bus Stop, Take-It-Away, Hands-First) with zero compromises.
  • Creating an embodied agent that exhibits true spatial agency—gazing at objects you hold and executing 3D object manipulation rather than acting as a passive 2D chatbot sidebar.
  • 100% zero paid API dependencies, keeping the application entirely free, private, and offline-capable.

📚 What I Learned

Spatial computing shouldn't replicate 2D monitors floating at infinite distances. By respecting human ergonomics and micro-gestures, VR transforms from an exhausting novelty into a deeply relaxing, tactile daily ritual.

🚀 What's Next for AetherDesk

  • Multi-user collaborative lap docking over WebRTC.
  • On-device Small Language Models (SLMs) via WebGPU for deeper semantic reasoning.
  • Packaging as an installable Progressive Web App (PWA) on the Meta Horizon Store.

Built With

Share this project:

Updates

Submission history