SpatialSurge

This is our app on the home screen

Inspiration

What it does

How we built it

What We Learned

Multi-panel spatial UI is genuinely better for reference-heavy tasks. The moment we had the steps panel and anatomy panel visible simultaneously alongside the coach, it was obvious this couldn't work on a phone. The trainee's eyes naturally scan between panels the same way a surgeon's eyes scan between monitors in an OR. PICO's Spatial SDK + Jetpack Compose is a surprisingly productive combo. We expected friction between Android's 2D UI framework and a spatial runtime. Instead, Compose handles each panel's content, PICO OS handles spatial placement, and they stay out of each other's way. ElevenLabs Scribe is underrated for domain-specific STT. Surgical terminology is a minefield for general-purpose speech recognizers. Scribe handled it with minimal errors out of the box. Snowflake's SQL API v2 is clean for lightweight mobile clients. We didn't need an ORM or connector library — just HTTPS POST with a JWT and a SQL string.

What's Next

Real 3D anatomy mesh. The Anatomy panel currently renders a 2D schematic via Compose Canvas. The next step is loading a .usd model through PICO's SpatialView — the TODO and integration point are already in AnatomyPanelActivity. Hand tracking. PICO 4 Ultra supports optical hand tracking. Pointing at a structure and asking, "What is this?" would make the coach interaction truly spatial. Snowflake event logging. Currently, Snowflake is read-only. Writing step transitions and voice queries to a TRAINING_EVENTS table would unlock post-session analytics dashboards. Streaming TTS. ElevenLabsTts uses the non-streaming endpoint. Switching to the streaming endpoint would shave ~400ms off perceived response time. Cadaver registration. Using PICO's spatial anchors to lock virtual overlays to a physical cadaver, so trainees can practice on real tissue with AI guidance.