degradation-watcher

Inspiration

Satellite constellations take millions of multispectral photos of Earth every week, yet human monitoring remains mostly manual and reactive. Agricultural land degrades quietly across weeks: irrigation systems fail, topsoil erodes, and drought stresses crops long before a farm manager can visually inspect hundreds of hectares on foot.

We built Degradation Watcher to explore multispectra remote sensing technique; so an asynchronous multi-agent pipeline that ingests free Sentinel-2 satellite imagery every 5 days, detects vegetation loss with multimodal vision, and drafts actionable mitigation reports before crop failure occurs.

What it does

Degradation Watcher is an event-driven system that monitors agricultural parcels and alerts operators to physical degradation:

• Automated Satellite Ingest: Tracks public Sentinel-2 L2A passes via the AWS Element84 STAC API, downloads Red, NIR, and True Color visual bands, and calculates NDVI vegetation health.
• Multimodal Temporal Diffing: Passes pairs of satellite captures (baseline vs. current revisit) into Gemini 3.6 Flash to compare pixel-level vegetation decline, soil exposure, and irrigation stress side-by-side.
• Composite Risk Scoring: Combines visual change severity (1–5 scale) with real-time weather stress from Open-Meteo, seismic proximity from the USGS Hazards API, and crop field metadata into a normalized 0–100 risk score.
• Autonomous Incident Drafting: Generates structured inspection reports with root-cause hypotheses, observed changes, and field-level remediation plans when risk thresholds cross tolerance levels.
• Dashboard: A Next.js 16 dashboard featuring interactive capture timelines, side-by-side temporal image diffing, and a parcel submission interface to register new farmland for monitoring.

How we built it

• Multi-Agent Architecture (Google ADK & Cloud Run): Built with @google/adk in TypeScript and deployed as microservices on Google Cloud Run:
  • Orchestrator Agent: Manages end-to-end task coordination, orchestrates temporal diffing image pairs, and invokes Gemini 3.6 Flash via the Google GenAI SDK.
  • Context Enricher Agent: Fetches historical and real-time soil moisture, precipitation, temperature, and seismic data from Open-Meteo and the USGS Earthquake API.
  • Risk Scorer Agent: Computes a deterministic weighted risk composite: Visual Change (40 pts), Weather Stress (30 pts), Seismic Activity (15 pts), and Crop Age/Exposure (15 pts).
  • Response Drafter Agent: Synthesizes agent context into formal remediation briefs for farm operators.
• Imagery Pipeline (Python & FastAPI): Ingests Sentinel-2 STAC metadata, crops regional bounding boxes, and automatically downsamples raw 200MB+ GeoTIFF tiles to 1024x1024 PNGs using Pillow to maintain sub-second browser rendering.
• Event Streaming & State (Cloud Pub/Sub & Firestore): Uses push subscriptions (imagery-ready, context-enricher-trigger, risk-scorer-trigger, response-drafter-trigger) to decouple agent execution asynchronously. Firestore stores parcel registries, degradation history, and alert queues.
• Web Application (Next.js 16 app): Uses Server Component reads, and Recharts to visualize risk score velocity and capture timelines.

Challenges we ran into

1. Payload Size & Memory Limits on Cloud Run: Raw multispectral GeoTIFF files from Sentinel-2 exceed 200MB per band. Passing raw TIF buffers between Cloud Run containers and downstream image optimizers caused out-of-memory crashes and request timeouts. We solved this by implementing in-memory thumbnail downsampling in the Python imagery service with Lanczos resampling, reducing file transfer size from 206MB to ~1MB per tile before Google Cloud Storage upload.
2. LLM Score Normalization: Early prompt iterations allowed multimodal models to output subjective scores outside expected ranges (e.g. severity 7/5 or confidence 9.0). We enforced strict JSON schemas in the prompt and added mathematical bounds clamping in the Orchestrator and Risk Scorer agents to prevent downstream score distortion.
3. Temporal Image Deduplication: Sentinel-2 revisits occur every 5 days. When running manual pipeline triggers across the same capture date, the orchestrator initially paired identical image URLs. We added deduplication logic to locate the immediate preceding unique capture date and pair distinct baseline images.

Accomplishments that we're proud of

• Reliable Multi-Capture Vision Diffing: Feeding two chronological satellite captures of the California Central Valley into Gemini 3.6 Flash yielded precise detection of seasonal harvest and drought-induced vegetative loss without hallucinated geography.
• Real-Time Event Architecture: The entire pipeline from Cloud Scheduler trigger to satellite download, multi-agent evaluation, and live dashboard rendering runs asynchronously through Pub/Sub in under 15 seconds.
• Production Asset Ingest: Built a functional UI allowing operators to register any global coordinates, assign crop types, define acreage and irrigation mechanisms, and enroll parcels into the automated monitoring cycle.

Other Data Sources

1. Sentinel-2 Multispectral Satellite Imagery (ESA / Copernicus): Free 10m-resolution multispectral imagery accessed via the AWS Element84 STAC catalog.
2. Open-Meteo Weather API: Real-time and historical soil moisture, precipitation, and ambient temperature data used by the Context Enricher agent.
3. USGS Earthquake Hazards API: Real-time seismic event proximity and magnitude data to evaluate physical ground disturbance.

What we learned

• Multimodal models perform significantly better at physical diffing when presented with paired images labeled chronologically (PREVIOUS CAPTURE vs. CURRENT CAPTURE) rather than analyzing a single snapshot against textual descriptions.
• Keeping raw geospatial data processing (raster calculation and reprojection) in lightweight Python microservices while keeping business logic and agent coordination in TypeScript ADK provides the best balance of speed, type safety, and maintainability on Cloud Run.

What's next for degradation-watcher

• Integrate radar backscatter to monitor soil moisture and ground subsidence through cloud cover and night passes.
• Integrate mitigation reports drafts
• Connect historical degradation trajectories directly to USDA county-level harvest predictions to forecast economic losses earlier in the growing season.

Built With

• bun
• docker
• fastapi
• firestore
• gemini-3.6-flash
• google-adk
• google-cloud
• google-cloud-pubsub
• google-cloud-run
• google-cloud-scheduler
• maplibre-gl
• nextjs
• open-meteo-api
• pillow
• python
• rasterio
• react
• recharts
• sentinel-2
• stac-api
• tailwindcss
• typescript
• usgs-api
• vertex-ai