Inspiration

Hand tracking is the most natural input technology in existence—everyone already owns their hands. Yet so many VR experiences treat hands like awkward controller stand-ins: point and pinch. We wanted the exact opposite: the hand pose IS the superpower. An open palm ignites a plasma blaster; a fist sparks twin katanas; a pinch draws a light bowstring; closing both fists extends energy claws.

We also wanted the player's physical space to be the front line. In SURGE, your living room is under siege. OBLIVAX, the Unmaker, has shattered reality by pulling the nine Anchors that stabilize the universe, and his dimensional breach begins in your actual room. Alien swarms tear through your detected walls, physics props shatter against your real coffee table, and every hero you master fights to reclaim reality before your room is erased.


What It Does

SURGE: Hands of Heroes is a controller-free mixed-reality (and VR) hero-combat game built natively for the Meta Quest browser. It features:

  • Nine Original Heroes & Nine Unique Hand Languages:

    1. ARC (Palm Blaster): Open palm facing forward fires continuous plasma bursts; pushing both hands forward channels a room-clearing chest beam.
    2. TALON (Energy Claws): Clench fists to extend razor-sharp energy claws; punch, slash, and skewer close-range foes.
    3. TEMPEST (Storm Hammer): Grip to summon Mjölnir-style hammer; throw it across your room, open palm to recall it, and call down lightning.
    4. ORION (Light Bow): Left hand stabilizes the bow riser; right hand pinches the string, pulls back, and releases energy arrows.
    5. WEAVER (Light Webs): Web-shooter hand pose shoots sticky tethers; whip your arm to fling ensnared aliens into walls.
    6. HAVOC (Plasma Pistol & Ordinance): Index-finger trigger squeeze fires semi-auto and burst munitions.
    7. RONIN (Twin Katanas): Clench dual fists to unsheathe energy blades; parry incoming attacks, slice, and cross wrists for a devastating X-Strike.
    8. PSION (Telekinesis): Open palm locks onto physics props or creatures; close fist to lift, and fling your arm to launch them across the room.
    9. VANGUARD (Energy Shield): Off-hand manifests an impenetrable barrier to deflect projectiles; swing and release to skip it across walls like a ricochet disc.
  • Real-Room Mixed Reality: Using WebXR plane detection, enemies emerge dynamically from real scanned walls. Ragdoll physics, dismembered parts, and thrown props bounce off real tables and floors with custom time-corrected Verlet collision.

  • Full 27-Mission Campaign: Nine planetary regions with 3 missions each, champion mini-bosses, shielded multi-phase regional overlords, and a climactic finale where all 9 heroes unite against Oblivax.

  • Endless & Daily Challenges: Infinite scaling enemy waves with boss milestones every 5th wave, plus a synchronized daily seed with global leaderboards for every hero.

  • Armory & Customization: 29 unlockable weapons, visual hero skins, and ability modifiers accessible straight from the spatial UI.

  • Zero Install, WebXR Freedom: Runs directly inside the Meta Quest browser at 72–90 fps with instant loading. Also fully responsive on mobile and desktop via our adaptive "puppet hand" simulator.


How We Built It

  • Engine & Core Architecture: Built with IWSDK (Immersive Web SDK) on Three.js (r185), TypeScript, and Vite. Architected around an Entity-Component-System (ECS) pattern with zero per-frame garbage-collection allocations.
  • Calibrated Hand Pose Classification: Hand gestures are tracked across all 25 WebXR joints per hand. Thresholds were empirically tuned using in-headset Quest 3 telemetry recordings (HAND DEBUG logs): open palms (1.8–2.0), fists (0.6–0.95), and precision pinches (sub-3cm delta).
  • Physics Engine: Engineered a custom lightweight Verlet integration solver for responsive real-time ragdolls, thrown projectiles, and spatial room colliders.
  • Mixed Reality Plane Integration: Dynamically queries WebXR plane detection to anchor portal breaches to physical walls and boundaries.
  • Spatial UI & Interaction: Menus support dual interaction modes: direct physical finger poke or gaze-and-pinch (compatible with Quest and Meta VR Glasses). Menus feature a physical drag handle to reposition the interface anywhere in your room.
  • Audio Design: Procedural Web Audio effects synthesized directly in code for weapons and impacts, combined with offline neural Kokoro-82M voiceovers and spatial 3D audio positioning.

Challenges We Faced

  1. The Lack of Physical Haptic Feedback on Bare Hands: Without physical controller vibration, impacts can feel ghost-like. We overcame this by synthesizing audiovisual heft: micro-frame hit-stops, chromatic aberration screen flashes, reactive enemy flinches, and punchy audio transients that trick the brain into feeling physical resistance.
  2. Hand Tracking Noise & False Positives: Differentiating a relaxed resting hand from a deliberate fist or pinch during frenetic combat is notoriously hard. We implemented velocity gating and temporal smoothing filters to eliminate accidental weapon triggers while preserving instant responsiveness.
  3. Strict Mobile XR Frame Budgets: Sustaining 72–90 fps on Quest 3 while rendering skinned meshes, real-world passthrough, and particle VFX required aggressive object pooling, instanced particle systems, and shader warmup during scene transitions.

Accomplishments We're Proud Of

  • 100% controller-free action game from start to finish—not a single controller button is ever required.
  • Nine distinct heroes that feel mechanically unique within five seconds of playing.
  • Seamless WebXR execution: players click a link and are in an immersive mixed reality battlefield in seconds without downloading a 10 GB APK.

What We Learned

Hands are far more expressive than buttons, but they demand rigorous calibration. Guessing gesture thresholds in code always fails; measuring real tracking joint angles across diverse hand shapes and lighting conditions is essential for intuitive XR gaming.


What's Next

  • Horizon Store PWA package submission for one-click library installation.
  • Optimization for Meta VR Glasses with dedicated gaze-and-pinch targeting.
  • Cross-device multiplayer co-op allowing Quest players to battle alongside mobile/tablet squad mates.

Built With

  • gltf
  • iwsdk
  • kokoro-tts
  • meta-quest
  • mixed-reality
  • node.js
  • passthrough-ar
  • playwright
  • react
  • remotion
  • three.js
  • typescript
  • vercel
  • verlet-physics
  • vite
  • vitest
  • web-audio-api
  • webgl
  • webxr
  • webxr-hand-input
  • webxr-plane-detection
Share this project:

Updates

Submission history