UrbanLens

UrbanLens is an AI agent that turns ordinary map imagery into thermal evidence, helping cities and campuses find heat-risk hotspots, rank what matters most, and decide what to inspect or fix first.

Comment

Inspiration

Urban heat is a problem people experience every day but rarely see clearly. Buildings, roads, and parking lots can trap heat in ways that increase energy consumption and impact public health, even when they look normal in satellite imagery.

Traditional solutions rely on expensive thermal drones and slow manual surveys, making it difficult to analyze cities at scale. Many of the most important issues—like inefficient roofing materials or heat-retaining infrastructure—are not visible to the human eye.

We wanted to solve this gap by asking:

What if an AI agent could investigate any location and generate thermal insight without requiring physical thermal sensors?

This led to UrbanLens, where we combine innovation with Agentic AI and our latest ThermalGen models to generate thermal predictions and uncover hidden environmental inefficiencies.

What it does

UrbanLens allows users to select a region on a map and run an intelligent environmental investigation.

Instead of simply showing a heatmap, the system performs a full agent-driven workflow:

Select map region → Capture satellite image → Generate thermal prediction (ThermalGen) → Detect heat hotspots → Score severity and anomaly → Rank findings → Recommend what to inspect or fix first

How we built it

UrbanLens is built as a multi-layered system:

Frontend

  • Interactive map interface using Google Maps
  • Region selection and submission
  • Toggle between satellite and thermal views
  • Displays ranked hotspots and recommendations

Backend

  • FastAPI-based service
  • Handles image capture and metadata
  • Runs thermal inference pipeline
  • Extracts hotspots and computes rankings

ThermalGen Models

  • Custom PyTorch-based RGB-to-thermal model
  • Generates thermal predictions from satellite imagery
  • Eliminates the need for expensive drone-based thermal surveys

Agentic AI Layer

  • Powered by Google Gemini via the Featherless API
  • Uses tool-calling and iterative reasoning
  • Treats ThermalGen as a tool within the agent
  • Combines multiple signals to investigate and prioritize results

Challenges we ran into

One major challenge was aligning all components across the pipeline. The satellite image, thermal prediction, and hotspot overlays had to remain perfectly synchronized. Even small mismatches caused incorrect visual outputs.

Another challenge was avoiding overclaiming results. ThermalGen produces relative heat patterns rather than exact temperatures, so we focused on comparative insights instead of absolute measurements.

We also struggled early on with defining a clear scope. The system initially risked being either:

  • too narrow (just a thermal visualization tool)
  • too broad (a general-purpose AI system)

We resolved this by focusing on a clear identity: an agentic environmental investigation system.

Accomplishments that we're proud of

  • Built a fully functional agentic AI system, not just a single model
  • Successfully integrated custom ThermalGen models into an AI reasoning pipeline
  • Replaced the need for expensive thermal drones with software-based predictions
  • Designed a system that moves beyond visualization into decision-making
  • Created a clean, intuitive interface for complex environmental analysis

What we learned

We learned that the most powerful AI systems are not about a single model, but about how different components work together.

By treating ThermalGen as a tool inside an agent, we were able to:

  • gather evidence
  • filter noise
  • prioritize meaningful results

We also learned how to design AI as an investigator, not just a generator—focusing on reasoning, context, and actionable insights rather than raw outputs.

What's next for UrbanLens

We plan to expand UrbanLens by:

  • Adding object and surface classification (roofs, roads, vegetation)
  • Integrating weather and environmental datasets
  • Improving planning recommendations (e.g., tree placement and cooling strategies)
  • Scaling to larger regions and improving real-time capabilities

Long-term, UrbanLens could support:

  • city planners
  • sustainability teams
  • infrastructure analysts

by providing fast, scalable alternatives to traditional thermal surveys and helping guide smarter environmental decisions.

Built With

Share this project:

Updates