Scorched Earth

Inspiration

Procedura's Scorched Nebraska imagines a persistent, collectively-built ruin of the present — real places, redesigned four hundred years from now by fire, flood, growth, dust, or mineral. We wanted to build the literal pipeline that makes that real: type in a real address, and watch a real building on a real map turn into its own ruin.

What it does

Type an address (or click the map). We geocode it and pull the real building footprint from OpenStreetMap — polygon, dimensions, orientation.
Upload a photo of the building, or pull one from Mapillary automatically.
Pick a World State — Reclaimed, Flooded, Scorched, Buried, or Petrified — on a Present ↔ Collapsed spectrum, or write your own description.
An AI model repaints the building into that state, keeping its exact architecture — same tower, same wings, same windows, only condition and surroundings change.
That image becomes a 3D mesh, normalized to real-world scale, and placed back on the map with an IoU rotation search against the real footprint, proportion-preserving scale fit, and neighbor collision checking.
Every generation persists — one row per generation, not per address — so a building can accumulate a whole timeline: Reality → Flooded → Reclaimed. "Propagate" spreads one building's World State to its real neighbors within a chosen radius, turning "we redesigned one building" into a neighborhood.
Anything the algorithm isn't confident about — an ambiguous footprint match, a bad rotation fit, a colliding placement — gets flagged and routed to a correction UI instead of silently guessing wrong.

How we built it

FastAPI backend, React + TypeScript frontend, MapLibre GL JS + deck.gl for the 3D map. Every external service is free, no credit card:

Geocoding & footprints: Nominatim + Overpass (OpenStreetMap) — we query both ways and relations, since VT's own Torgersen Hall, Newman Library and Kelly Hall are all mapped as multipolygon relations that a naive query misses entirely.
Photo input: manual upload (required) plus an optional Mapillary auto-fetch, which we found returns non-deterministic result counts for identical queries — so it retries before reporting "no coverage."
Image generation: Google Gemini's image model, falling back to a FLUX.1 Kontext space on Hugging Face.
3D reconstruction: Stable Fast 3D, with TripoSR as a fallback — including a version that runs entirely locally on Apple Silicon with no external quota at all.
Placement: our own geometry engine — IoU-scored rotation search at four candidate angles, oriented (not axis-aligned) width/depth measurement, polygon-intersection collision detection, and a rotation-fit ceiling that accounts for how much of a real footprint a single-photo reconstruction can actually cover.
Persistence: Supabase Postgres, with a transparent SQLite fallback for local development.
Reliability: every external response is cached to disk, four Overpass mirrors are tried in sequence with backoff, and the demo is fully pre-baked end-to-end so judging never depends on a live API responding.

Four of us split the fourteen-module spec — entry pipeline, AI generation, placement geometry, and persistence/map/UI — and built against one shared API contract so the pieces snapped together.

Challenges we ran into

The public services we depend on are genuinely unreliable. All four Overpass mirrors went down at different points during the build, and both free image-generation tiers hit zero quota. We built caching, mirror fallback, and full artifact pre-baking specifically because we couldn't trust anything to be up during judging.
The "obvious" geometry approach was wrong twice. A flat IoU threshold for rotation correctness flagged every real building, because a rectangle can never fully cover an L-shaped or bridged footprint — the check had to be relative to what's achievable, not absolute. Axis-aligned bounding-box collision detection produced false positives on diagonal buildings whose boxes overlapped while the buildings themselves didn't — we switched to real polygon intersection.
Real-world testing found real-world bugs. Running the full pipeline against an actual generated mesh caught a units mismatch in our own placement code before it ever reached a judge.

What we learned

That the gap between "should work" and "works against real messy data" is where all the actual engineering is. Every one of our confidence thresholds and fallback paths exists because something we assumed would just work, didn't, the first time we ran it against something real.

What's next

Live pitch and correction workflow polish, a second set of pre-generated demo buildings, and wiring in a Hugging Face token with fresh daily quota so live generation is demoable end to end on stage, not just pre-baked.