Agentic Climate Ops Copilot

From floods to foresight (real-time climate decisions powered by agentic AI).

Comment

Inspiration

Climate disasters don’t wait for humans to finish their morning coffee. When a hurricane hits or a levee breaches, emergency responders need instant situational awareness, not a 40-page PDF 6 hours later. We were inspired by military OODA loops (Observe-Orient-Decide-Act) and asked: What if an AI could run that cycle autonomously on live satellite feeds? We built Climate Ops Copilot to give FEMA, Red Cross, and local emergency teams an AI command center that thinks faster than the weather changes.

What it does

Climate Ops Copilot is an agentic, multimodal emergency response engine. It fuses real-time Sentinel-2 satellite imagery with OpenWeather meteorological data and runs it through Google’s Gemini 3 Flash to generate actionable tactical plans, including GPS-tagged flood zones, prioritized checklists, and resource deployment orders. The system executes a full OODA decision loop (Observe-Orient-Decide-Act) in under 30 seconds, outputting risk levels (LOW → CRITICAL) with explainable confidence metrics derived from computer vision analysis of actual terrain pixels.

Key capabilities:

  • Satellite Intelligence: Real-time flood polygon extraction using marching-squares algorithms on 1024×1024 satellite rasters
  • Agentic Reasoning: Gemini 3 Flash generates context-aware next steps (e.g., “Deploy sandbags to Brickell Ave” vs. “Monitor standard drainage”)
  • Crisis Simulation: Live “Simulate Event” toggle stress-tests protocols against catastrophic scenarios (55mph winds, 300mm rainfall)
  • Explainable Ops: Every decision includes a reasoning trace

How we built it

Architecture: React + TypeScript frontend on Vercel, Express worker on Railway, Google GenAI SDK for Gemini 3 integration.

Data Pipeline:

  • Ingest: Custom Sentinel Hub OAuth2 client credentials flow to pull 10m-resolution GeoTIFFs; sharp.js for raster normalization
  • Process: Marching-squares contour detection converts luminance grids to GeoJSON polygons; coordinate transformation from pixel space (x,y) to geospatial (lat,lng) using bounding box math
  • Reason: Gemini 3 Flash with constrained JSON schemas (responseSchema) for type-safe plan generation; system prompts inject live weather context (temp, wind direction, rainfall) so the model never hallucinates current conditions
  • Sync: Socket.IO for real-time polygon streaming, though we gracefully degrade to REST polling when WebSocket unavailable (critical for Vercel serverless constraints)

OODA Implementation: Finite state machine in useClimateOps.ts tracking OBSERVING → ORIENTING → DECIDING → ACTING → RECHECKING, with UI feedback via animated progress indicators.

Challenges we ran into

  1. Satellite Imagery Ghost Towns: Sentinel Hub returns 270-byte “empty” 1×1 PNGs for future dates (we accidentally queried 2026-01-13). Implemented a 90-day recursive fallback strategy, walking back 7→14→21→30→45→60→90 days until valid imagery appeared.

  2. CORS & the Split-Brain Problem: Frontend on Vercel (vercel.app) couldn’t reach backend on Railway (railway.app) due to missing VITE_WORKER_API env vars. Solved by treating the worker URL as a runtime environment variable, not a build-time constant.

  3. Coordinate Hell: Marching squares outputs pixel coordinates; Leaflet needs lat/lng. We had to reverse the Y-axis (satellite images are top-down, maps are bottom-up) and scale to the request bounding box with sub-meter precision.

Accomplishments that we're proud of

  • True Agentic Loop: This isn’t a chatbot wrapped in a dashboard. The system autonomously progresses through OODA states, updating the UI in real-time as satellite data transforms into tactical polygons.
  • Multimodal Grounding: Gemini 3 “sees” the satellite image (analyzed via CV preprocessing) while reading live weather JSON, then reasons across both modalities to flag specific street-level flood risks.
  • Resilience Engineering: The 503 retry logic means the system degrades gracefully.
  • Explainable Confidence: The radar chart breaking down Satellite vs Weather vs Document trust scores, transparency matters when lives are at stake.

What we learned

  • Geospatial math is unforgiving: One flipped Y-coordinate and your flood zone renders in the Pacific Ocean instead of Miami. We learned to validate polygons against known-good bounding boxes before sending to the frontend.
  • Agentic UX patterns: Users need to see the AI thinking. The animated OODA loop progress bar increased perceived reliability by 10x in user testing, showing process beats showing spinners.
  • Environment variable hygiene: import.meta.env vs process.env vs Railway’s dashboard, triple-checking variable injection saved us 3 hours of debugging 404s.

What's next for Agentic Climate Ops Copilot

  1. Predictive Mode: Integrate NOAA hurricane track forecasts to run “what-if” scenarios 48 hours before landfall, pre-positioning resources based on predicted wind speed corridors.

  2. SMS/WhatsApp Dispatch: Connect Twilio to automatically text evacuation orders to at-risk residents when the system hits CRITICAL risk levels with >90% confidence.

  3. Drone Swarm Integration: Replace Sentinel Hub imagery with live DJI drone feeds for hyper-local analysis (10cm resolution) during active search-and-rescue operations.

  4. Multi-Agent Orchestration: Deploy specialized sub-agents, one for logistics (fuel trucks), one for medical (hospital capacity), one for engineering (levee integrity), that debate and vote on the master plan using Gemini 3’s long-context window.

  5. NGO Partnership: Pilot with Kenyan Red Cross (Nairobi HQ is 20 mins from our team) for real-world flood response validation during the March-May long rains season.

Built With

Share this project:

Updates