TerraTwinSOS

Inspiration

When catastrophic disasters strike, whether structural collapse following a severe earthquake, sudden monsoon flash flooding, or urban fires, first responders and trapped survivors face a fatal bottleneck: spatial blindness. In informal settlements or disaster-damaged zones, traditional street addresses don't exist, and raw GPS coordinates are virtually useless to an incident commander trying to figure out which stairwell collapsed or where a rescue ladder needs to reach. Inspired by digital spatial anchors, TerraTwin SOS reverses the concept for disaster survival: anyone can drop an emergency beacon that instantly snaps to an AI-reconstructed 3D digital twin, calculating exact human-readable addresses, building structure IDs, discrete floor levels, and real-time hazard risks.

What it does

TerraTwin SOS is an end-to-end multi-hazard digital twin pipeline and emergency dispatch ecosystem:

Satellite AI Segmentation: Ingests raw sub-meter satellite imagery and extracts crisp building footprints using an optimized PyTorch U-Net with an outstanding test IoU of 0.8062. Algorithmic Addressing: Clusters structures into neighborhood blocks via DBSCAN, numbers them in natural reading order, and snaps them to OpenStreetMap road vectors to generate human-readable physical addresses. Mobile SOS Beacon (PWA): A one-tap citizen web app featuring a Web Audio API acoustic rescue locator beacon that pulses a dual-tone chirp (880Hz / 1320Hz) to guide search dogs and rescue teams under rubble. Mission Control Dispatcher Hub: Includes a real-time 3D Hydraulic Flood Inundation Simulator (with surge controls from 0.0m to +10.0m), live USGS earthquake feeds, and MMI shakemaps.

Volumetric Floor Studio: Built with Three.js, allowing operators to separate floor slabs in mid-air, toggle FLIR thermal vision, or use X-ray wireframe modes.

Tactical Rescuer HUD: Provides field teams with turn-by-turn routing, distance countdowns, and an operational status stepper (Target SOS \rightarrow En Route \rightarrow On Scene \rightarrow Evacuating).

How we built it

Computer Vision & Backend: PyTorch 2.5, segmentation_models_pytorch, and CUDA 12.1 for training the ResNet34 U-Net model with mixed-precision autocast. Powered by a FastAPI server with thread-safe in-memory spatial indexing and sub-millisecond reverse geocoding.

Geospatial & Topography: Utilized rasterio, geopandas, shapely, and AWS Open Data Terrarium Digital Elevation Models (DEM) with bilinear interpolation and LRU memoization to establish true Mean Sea Level height precision.

Frontend & 3D Visualization: MapLibre GL JS for map layers and a Three.js WebGL volumetric studio with custom extrusion shaders for interactive multi-floor inspection.

Challenges we ran into

The fp16 Autocast NaN Bug: Standard PyTorch float16 autocast caused immediate NaN forward logits with our ResNet34 encoder; we identified the numerical instability and transitioned smoothly to bf16 autocast without gradient scaling.

Online open source data for 3d Google maps were unavailable

DEM Ground Elevation Discrepancies: Early iterations lacked true topographic anchoring until we integrated AWS Terrarium DEM data to resolve ground heights to within \pm 1.0\text{m} accuracy.

Multi-Floor 3D Separation: Standard 2D footprints cannot isolate individual floors, so we built a custom geometry engine that extrudes separate habitable floors and 18cm concrete structural slabs for dynamic explosion views.

Accomplishments that we're proud of

The 40× Performance Gap: Proving our model's credibility against a zero-shot pretrained baseline (which scored a dismal 0.02 IoU) by hitting a soaring 0.8062 Test IoU and 0.8927 F1 Score through specialized satellite fine-tuning.

Blazing Fast Optimizations: Reducing payload sizes by 86% via GZip compression and cutting DEM point elevation lookup latency down to 1.07 ms (>1,400× speedup).

Complete Privacy-First Architecture: Implementing absolute zero PII collection alongside an automated 24-hour incident expiration purge.

What we learned

We learned the hard way that off-the-shelf computer vision models trained on Western urban datasets fail completely when deployed in arid or regional environments—making domain-specific fine-tuning and dataset-wide normalization essential. We also discovered how critical lightweight, zero-install PWAs are for field usability during high-stress emergency scenarios.

What's next for TerraTwinSOS

Autonomous Drone Path Planning: Utilizing the Unity URP digital twin export to train autonomous UAV agents for obstacle avoidance and survivor scouting in collapsed building rubble.

Satellite Change Detection: Automatically comparing pre- and post-disaster satellite imagery passes to instantly flag collapsed structures and blocked roads.

Indoor Sub-Meter Localization: Integrating smartphone barometric pressure sensors and local beacons to pinpoint exact room-level positions within multi-story complexes.