Leto — Making Building Retrofits Accessible

The all-in-one energy consultant that combines building data, physics-based simulations, and financial analysis to make energy-efficient upgrades more accessible.

Inspiration

Buildings account for roughly 27% of global energy-related CO₂ emissions. The interesting thing is that we already have much of the technology needed to reduce these emissions. Solar panels, heat pumps, better insulation, and energy-efficient lighting are all commercially available.

The problem is figuring out which upgrades actually make sense for a particular building.

Property owners often have to spend hundreds or thousands of dollars just to evaluate their options, before making any actual improvements. And even then, the recommendations depend heavily on the building's geometry, existing equipment, location, and energy consumption.

We wanted to change that.

What if we could use existing building data to conduct an initial energy audit without having to physically visit the property?

More importantly, what if we could use actual engineering calculations to determine which improvements make financial and environmental sense, and help owners take the next steps toward implementing them?

That's why we built Leto.

What it does

Leto is an AI-powered building energy consultant that helps property owners identify and evaluate energy-efficient upgrades.

Users select a property, specify their budget, and optionally provide utility bills. Leto combines three sources of building information:

  • CoStar: Property records, including building age, floor area, occupancy, and use.
  • Matterport: Interior 3D scans providing information about rooms, windows, lighting, and existing equipment.
  • Vexcel: Aerial imagery and roof geometry for evaluating solar potential and exterior improvements.

We combine these datasets with local weather, solar irradiance, utility rates, and regional emissions factors to construct a physics-based model of the building.

From there, Leto can:

  1. Recommend energy-efficient upgrades: Evaluate improvements such as solar panels, lighting, windows, and equipment replacements based on installation costs, energy savings, and the user's budget.

  2. Quantify financial and environmental impact: Estimate electricity savings, CO₂ reductions, available incentives, payback periods, and other financial metrics.

  3. Help implement recommendations: Generate draft retrofit contracts, solar documentation, and commercial C-PACE financing reviews where applicable.

We also wanted users to understand where the recommendations come from. Leto includes an interactive building walkthrough, roof visualizations, and an audit trail showing the assumptions and data behind its calculations.

Our goal wasn't just to tell people to install solar panels. We wanted to explain whether solar panels actually make sense for their building, where they could go, how much energy they might generate, and whether the investment is worthwhile.

How we built it

We divided Leto into four main components: building data, computer vision, physics-based simulations, and an AI agent that connects everything.

Building Data and Computer Vision

We integrated all three CoStar track datasets into a single building model. Matterport provides interior geometry and panoramic imagery, Vexcel provides aerial imagery and roof information, and CoStar supplies property and occupancy data.

We also used Claude to analyze Matterport panoramas and identify visible building features, including window types, lighting, HVAC equipment, water heaters, and electrical panels.

Each observation is recorded with its source and an associated confidence level, allowing the model to distinguish between known building characteristics and assumptions.

Physics Engine

This was one of the most technically interesting parts of the project.

Rather than relying on an LLM to estimate energy savings, we built a dedicated physics engine that performs the calculations.

Our model runs an hourly thermal simulation covering all 8,760 hours of the year, incorporating building geometry, weather conditions, solar gains, and internal energy loads.

For solar installations, we calculate the sun's position and irradiance on individual roof planes, accounting for shading, panel placement, and system losses.

We also wrote our physics specifications before implementing the code, documenting the equations and building tests around worked examples. This allowed us to verify the calculations independently of the AI agent.

AI Agent

We built our AI consultant using LangChain deepagents and LangGraph, with specialized subagents responsible for different parts of the workflow:

  • Scout: Retrieves building and property information.
  • Surveyor: Analyzes interior scans and constructs the building model.
  • Physicist: Runs energy simulations and evaluates upgrades.
  • Financier: Analyzes costs, incentives, and financing.
  • Counsel: Generates draft legal documents.
  • Reviewer: Checks the final report against calculated results.

The main agent coordinates these tasks but isn't allowed to invent numbers or perform its own arithmetic. All numerical results must come from the underlying analysis tools.

We also built a faster analysis pathway for users who want an initial plan without running the full agent workflow.

Tech Stack

  • Frontend: React, TypeScript, Vite, three.js, Matterport SDK
  • Backend: Python, FastAPI
  • AI: Claude, LangChain deepagents, LangGraph
  • Data: CoStar, Matterport, Vexcel, Open-Meteo, PVGIS, EIA, EPA eGRID

Challenges we ran into

One of our biggest challenges was determining whether our predictions were actually accurate.

It's relatively easy to create a simulation that produces reasonable-looking numbers. It's much harder to validate those numbers against real buildings.

We tested our model against metered electricity data from the Building Data Genome Project 2, Los Angeles building benchmarking records, and independent solar estimates from PVGIS.

One particularly promising result came from calibrating the model using actual utility bills. Across 44 buildings that successfully calibrated, median prediction error on an unseen year decreased from 18.1% to 4.1%. However, calibration didn't succeed for every eligible building, and predictions without bills remain less reliable.

For solar, our estimated annual generation was within 10% of PVGIS across 76 scanned Los Angeles roofs, with a median difference of approximately 2%. This validated agreement between the models, although actual installation performance still needs to be tested.

We also encountered several integration challenges, including overlapping roof segmentation masks, inconsistencies between Matterport scans and property records, and issues with building geolocation.

Finally, coordinating four developers working across multiple interconnected components was a challenge of its own. We used a shared GitHub issue as a communication board for our team and coding agents, with clearly defined ownership of different components.

Accomplishments that we're proud of

Our biggest accomplishment was building something that combines AI with actual engineering analysis rather than treating AI as a substitute for it.

We successfully integrated CoStar, Matterport, and Vexcel into a single workflow that connects physical building characteristics with energy simulations, financial recommendations, and document generation.

We're particularly proud of:

  • Building a physics engine that simulates a building's energy consumption across an entire year.
  • Integrating three complementary building datasets into one model.
  • Developing a multi-agent system where numerical results are generated and checked by dedicated tools.
  • Achieving strong agreement with PVGIS for solar generation estimates.
  • Validating our predictions against independent datasets and using the results to improve the product.

Most importantly, we built an application that makes complex engineering analysis understandable and actionable for property owners.

What we learned

The biggest thing we learned was that a physically grounded model is only as good as its inputs.

Even with established physics equations, assumptions about occupancy, insulation, equipment efficiency, and building operation can significantly change the results.

We also learned how important it is to separate AI reasoning from numerical computation. Giving each agent a clearly defined responsibility and keeping calculations in dedicated code made the system easier to test and debug.

Finally, writing our physics specifications before the implementation paid off. It reduced ambiguity, simplified testing, and gave us a much stronger foundation for the application.

What's next for Leto

Our first priority is improving model accuracy across different building types and incorporating utility-bill calibration more consistently. We also want to validate individual retrofit savings against completed projects and improve our uncertainty estimates.

Beyond that, we want Leto to work from just a building address, even when a Matterport scan isn't available. This would make the platform accessible to a much larger number of properties.

One particularly interesting extension is portfolio-level analysis. Instead of evaluating buildings individually, real estate investors could use Leto to identify which properties offer the greatest potential financial and environmental returns from energy-efficient upgrades.

We also want to expand our legal document generation to additional states and compare our recommendations against professional energy audits and contractor quotes.

Ultimately, our goal is to make building energy assessments more accessible, affordable, and actionable.

We already have the technology to make buildings significantly more efficient. We want to make it easier for people to figure out where to start.

Built With

  • claude
  • computer-vision
  • costar
  • deepagents
  • eia
  • epa-egrid
  • fastapi
  • langchain
  • langgraph
  • matterport
  • open-meteo
  • physics-simulation
  • pvgis
  • python
  • react
  • solar-energy
  • three.js
  • typescript
  • vexcel
  • vite
Share this project:

Updates

Submission history