EcoEquity Agent: Live Geospatial AI for Climate Justice

A multimodal AI agent that "sees" satellite data in real-time to detect urban heat islands and advocate for environmental equity via live voice interaction.

Comment

Inspiration

Urban heat islands are not just an environmental issue; they are a matter of social equity. Many low-income neighborhoods lack the green canopy necessary to combat rising temperatures.

EcoEquity Agent was inspired by the need to democratize geospatial data, turning complex satellite imagery into actionable, real-time conversations for climate justice.

What it does

EcoEquity Agent is an autonomous Live Geospatial Gemini Agent that "sees" cities from space and reasons in real time.

Users interact via voice or text, and Gemini autonomously orchestrates tools and multimodal inputs to:

  • Execute Real-Time Scans: Dynamically calls satellite APIs (e.g., Sentinel-2) to detect low vegetation areas using NDVI analysis
  • Compile Ecological Audits: Aggregates surface temperature, vegetation health, and environmental signals in real time
  • Reason & Strategize: Uses multimodal reasoning to synthesize visual + environmental data and generate targeted reforestation strategies

How we built it

We leveraged a high-performance, multi-model architecture on Google Cloud, allowing dynamic selection of the optimal neural core based on task complexity and latency requirements:

  • Brain: A flexible Gemini-powered system utilizing:

    • Gemini 3.1 Pro (Preview) for advanced reasoning and deep analysis
    • Gemini 3.0 Flash (Preview) for ultra-fast, low-latency multimodal responses
    • Gemini 1.5 Pro / Flash for stable fallback and task balancing

  • Vision: Custom integration with satellite data providers, processed through Vertex AI

  • Tool Use & Grounding: Gemini dynamically orchestrates external tools (satellite APIs, datasets) and uses Vertex AI Search Grounding to ensure outputs are backed by verified climate data

  • Infrastructure: The backend is fully containerized and deployed on Google Cloud Run for real-time scalability

Challenges we ran into

Synchronizing real-time visual map frames, tool execution, and a live voice agent was our most complex engineering challenge.

We optimized the pipeline to send sector-specific geospatial scans to Gemini, enabling context-aware insights (such as identifying a specific street's canopy gap) in under 2 seconds.

Accomplishments that we're proud of

  • Real-Time Multimodal Sync: Successfully integrating a live vision feed from satellite imagery with a Gemini-powered voice agent, achieving near-zero latency

  • Impactful Design: Creating a high-fidelity Cyber-Dark HUD that makes complex environmental data (NDVI, thermal spikes) accessible and intuitive

  • Vertex AI Integration: Transitioning the intelligence core to Google Cloud Vertex AI, ensuring real-world grounding and enterprise readiness

  • Autonomous Reasoning: Moving beyond a chatbot to a true AI Agent capable of independently identifying heat risks and generating actionable reforestation strategies

What we learned

We discovered the immense power of Multimodal Agentic Workflows.

When an AI can see, reason, and act in real time, it evolves into a digital twin of the environment—capable of continuous analysis and intelligent decision-making.

What's next for EcoEquity Agent: Live Geospatial AI for Climate Justice

  • Global Expansion: Expanding satellite pipelines for worldwide coverage
  • Predictive Simulation: Simulating the cooling impact of green infrastructure before implementation
  • Community Feedback Loop: Integrating resident-reported heat data with satellite telemetry in real time
  • Mobile Integration: Building a mobile AR interface to visualize AI-generated climate strategies directly on streets

Impact

EcoEquity Agent transforms satellite intelligence into real-time, agent-driven climate action—empowering communities to combat urban heat inequality through data-driven decisions.

Built With

Share this project:

Updates