Inspiration
Buildings produce roughly 34–39% of global energy-related CO₂ emissions, yet many property owners never make the upgrades that could reduce their environmental impact. The problem isn't necessarily a lack of interest in sustainability. It's that owners don't have an easy way to understand what improvements their properties can support, whether those improvements are financially worthwhile, or how much risk is involved.
We wanted to understand this problem from the perspective of actual property owners, so we reached out to Krishna Chavan, Founder of Sneha Coastlines, a business involved in property management in Costa Rica that also owns property there.
Krishna highlighted several challenges his business faces: evaluating a property's environmental impact, determining whether sustainable upgrades like solar panels and energy-efficient appliances are worth the investment, finding applicable tax incentives, and comparing properties using both financial and environmental factors.
He described the need for "a centralized tool to evaluate sustainability, estimate potential savings, and support smarter real estate decisions."
That conversation became the foundation for HawkLens.
We asked a simple question: What if an AI agent could investigate a building the way an engineer would, from above and from the inside, determine which sustainable improvements are actually feasible, calculate their financial impact, and build an investment strategy around the owner's goals?
What it does
HawkLens is a multimodal AI agent that turns real estate properties into actionable sustainability investment plans.
Instead of simply displaying property information or generating generic sustainability recommendations, HawkLens investigates each building, identifies potential improvements using spatial and visual data, evaluates their financial and environmental impact, and determines the most cost-effective path toward an owner's emissions target.
The system combines all three CoStar data layers with a multi-agent architecture, a Monte Carlo financial engine, and Capital One's Nessie API to take a property owner from initial assessment to financing.
1. Survey: Understand the building
The owner selects a property portfolio and defines a sustainability goal.
HawkLens gathers property information from the CoStar Data Platform and analyzes Vexcel aerial imagery, including overhead and directional oblique views, to understand the building's physical characteristics.
Where Matterport scans are available, HawkLens also uses interior spatial data, including room boundaries, wall graphs, panoramas, and 3D meshes.
Combining these sources gives the system a much more complete understanding of each property than relying on a property listing or satellite image alone.
2. Investigate: Identify feasible improvements
A team of specialized AI agents investigates potential sustainability upgrades.
The aerial inspector examines roofs and facades for usable solar area, shading, existing panels, and glazing. The interior inspector analyzes available Matterport data for features such as lighting and HVAC.
A site engineer agent combines these findings with the property's records to determine which upgrades are feasible and estimate their potential scale, such as the amount of solar capacity a roof can support or the square footage of insulation that could be added.
A separate verifier agent challenges the inspectors' findings by checking claims against their source images. Claims with confidence scores below 0.7 are sent back for further investigation with a specific question.
If the system cannot establish something reliably, it reports it as Unknown rather than inventing information. Unverified findings are excluded from the optimization process.
3. Evaluate: Calculate the financial and environmental impact
Every identified upgrade is evaluated through 10,000 Monte Carlo simulations covering variables such as energy prices, carbon fees, incentives, and equipment lifespans.
The engine calculates financial and environmental metrics for each improvement, including net present value (NPV), probability of financial loss, conditional value at risk (CVaR), and cost per tonne of CO₂ avoided.
HawkLens presents these results through an abatement curve, showing which upgrades reduce emissions while also making financial sense.
This allows owners to compare improvements based on more than their initial installation cost.
4. Optimize: Build the investment strategy
Owners can set a specific emissions reduction target, such as reducing a portfolio's emissions by 40%.
Rather than recommending upgrades individually, HawkLens uses an optimization engine to identify the lowest-cost combination of improvements that achieves the owner's goal.
The resulting plan includes the recommended upgrades, expected costs, probability of achieving the emissions target, and worst-case financial outcome.
This makes it possible to compare different investment strategies while accounting for uncertainty and financial risk.
5. Finance: Turn recommendations into action
Once the owner approves the investment strategy, HawkLens connects to Capital One's Nessie sandbox to execute the financing workflow.
The finance agent creates a dedicated Retrofit Reserve account, funds it, and books contractor bills through the Nessie API.
The system also automatically generates an investment memo using the financial engine's actual outputs, giving owners a consolidated breakdown of their recommended investments, projected returns, and associated risks.
The result is a complete workflow from property assessment to an actionable, financially backed sustainability plan.
How we built it
Agent Architecture
We built HawkLens using a LangGraph multi-agent architecture consisting of an orchestrator, records analyst, aerial inspector, interior inspector, site engineer, verifier, pricing agent, finance agent, and UI composer.
Each agent has a specialized responsibility, allowing the system to break a complex real estate assessment into smaller tasks that can be independently executed and validated.
All agent outputs are validated using Pydantic schemas. The UI composer can only select from registered widgets with valid properties, ensuring that dynamically generated interfaces remain structured and predictable.
Gemini API
Gemini powers the multimodal reasoning behind our aerial inspector, interior inspector, verifier, and UI composer.
It analyzes overhead imagery, directional oblique images, and Matterport panoramas to identify building features and potential sustainability improvements.
The verifier uses visual evidence to challenge the inspectors' findings rather than blindly accepting model-generated recommendations.
CoStar: Integrating All Three Data Layers
One of the most important parts of HawkLens is how we combine the three CoStar data layers to understand real buildings.
- CoStar Data Platform: Provides building profiles and property information used throughout the sustainability analysis.
- Vexcel: Provides overhead and N/E/S/W oblique imagery. These images are analyzed to identify physical building characteristics, potential issues, and opportunities for sustainable improvements.
- Matterport: Provides interior spatial information, including room boundaries, wall graphs, panoramas, and 3D meshes. This data powers our interactive 3D building visualizations.
These layers serve complementary purposes. The Data Platform and Vexcel imagery inform the sustainability algorithm and help our agents identify issues, while Matterport allows us to represent the actual building as an interactive 3D model.
Users can rotate and explore their buildings and see the potential issues and improvements identified by HawkLens directly on the visualizations, connecting the agent's recommendations to the physical property.
Quantitative Optimization Engine
We built a financial simulation and optimization engine using NumPy and PuLP with CBC.
NumPy runs Monte Carlo simulations to calculate per-upgrade NPV, probability of loss, CVaR, and cost per tonne of CO₂ avoided.
PuLP then solves the investment optimization problem, finding the lowest-cost combination of upgrades that satisfies the owner's emissions target while accounting for financial risk.
We use the Rockafellar-Uryasev formulation to keep CVaR linear and compatible with our optimization solver.
The optimizer runs approximately 1,000 scenarios for faster interactions, while the financial charts use 10,000 simulations. We also use a fixed random seed to ensure results remain consistent between interactions.
Every input range has a documented source.
Capital One Nessie
Nessie's mock financial data provides the financial foundation for each property's energy spending and the financing workflow.
Monthly utility bills establish the building's baseline energy expenditure. Once an owner approves a plan, the finance agent uses Nessie to create and fund the Retrofit Reserve account and book contractor bills.
This connects the sustainability analysis to an actual financial workflow rather than stopping at recommendations.
Frontend and Infrastructure
Frontend: React (Vite), Recharts, and three.js through react-three-fiber. Buildings are extruded from their footprints and textured with aerial imagery. The hero building also renders its real Matterport mesh.
Backend: FastAPI, deployed on a Railway instance.
Domain: hawklens.vip, registered through GoDaddy Registry.
Challenges we ran into
Making AI-generated findings trustworthy
One of our biggest challenges was getting reliable information from visual analysis. Early roof readings could be confidently wrong, which was especially problematic because incorrect measurements could influence the financial recommendations.
We addressed this by introducing a dedicated verifier agent. Every visual claim receives a confidence score, and claims below 0.7 are returned to the inspector with a specific question for further investigation.
We also made sure that unconfirmed findings never reach the optimizer. If information cannot be established, the system reports it as Unknown instead of making assumptions.
Keeping financial calculations honest
We wanted to ensure the AI could explain financial results without inventing or modifying the numbers- no hallucination.
The LLM is responsible for the narrative surrounding the results, while widgets and investment memos receive their numerical values directly from deterministic tool outputs.
We also implemented a guardrail that checks whether the numbers in generated investment memos match the underlying financial engine.
Making the optimization engine fast enough to feel interactive
Running 10,000 simulations for every optimization would make the application feel slow.
We separated the computational workloads, using approximately 1,000 scenarios for the optimizer and 10,000 for financial charts. A fixed seed keeps visualized results consistent between interactions without sacrificing the depth of the financial analysis.
Connecting physical building data to financial decisions
Another challenge was translating visual observations into something usable for investment decisions.
Identifying a large roof is not enough to recommend solar panels. We needed to connect the available roof area, shading, existing infrastructure, energy assumptions, and financial projections.
Our site engineer agent bridges this gap by turning observations from the different data sources into structured, measurable upgrade opportunities that the financial engine can evaluate.
Accomplishments that we're proud of
- Combining all three CoStar data layers: Built a system that uses property records, aerial imagery, and interior spatial data together to understand real buildings and evaluate their sustainability.
- A multimodal agent that investigates and verifies: Created specialized agents that identify potential building improvements, challenge uncertain findings, and provide visual evidence rather than relying on unsupported assumptions.
- Interactive 3D property visualizations: Integrated real Matterport data into interactive building models, allowing owners to explore their properties and see the issues identified by the agents.
- A risk-aware financial engine: Implemented Monte Carlo simulations and mathematical optimization to find cost-effective investment strategies under uncertainty.
- A complete financing workflow: Connected the sustainability planning process to Capital One's Nessie sandbox, allowing approved plans to move into account creation, funding, and contractor billing.
- End-to-end functionality: Connected property assessment, visual investigation, financial modeling, optimization, and investment memo generation into one application.
We are especially proud that HawkLens doesn't stop at telling property owners what could be improved. It helps them understand why an improvement makes sense, whether they can afford it, and how to move forward.
What we learned
Multimodal AI becomes much more valuable when different data sources complement each other. Aerial imagery can reveal opportunities that property records cannot, while interior spatial data adds another dimension to understanding a building. Combining these sources creates a much more complete picture than any one layer could provide independently.
Agentic AI needs verification, not just autonomy. We learned that giving agents the ability to make decisions is only useful when there are mechanisms to challenge their conclusions. Our verifier agent and Unknown handling taught us that a system should be comfortable acknowledging uncertainty rather than confidently producing an incorrect answer.
Sustainability decisions are also financial decisions. Property owners need more than estimated carbon savings. They need to understand installation costs, payback periods, incentives, uncertainty, and how individual improvements fit into a larger investment strategy. Connecting environmental impact to financial outcomes makes sustainability more actionable.
Deterministic tools and AI should have different responsibilities. We found that the most reliable approach was to let multimodal AI interpret buildings and investigate potential improvements, while dedicated mathematical tools handle financial calculations and optimization. Separating these responsibilities made our system easier to validate and trust.
Finally, talking to a real property management business helped us understand that these aren't just technical problems. Property owners genuinely need a centralized way to compare environmental and financial considerations when making real estate decisions.
What's next for HawkLens
We see HawkLens evolving into a more comprehensive sustainability intelligence platform for real estate owners, investors, and property managers.
- Expand property coverage: Improve our agents and broaden the range of buildings and geographic locations they can reliably analyze. We want to move beyond our initial set of properties and support larger, more diverse portfolios.
- Building-specific specialization: Develop specialized agent teams for different property types, such as residential buildings, commercial offices, hotels, and industrial facilities. Each type has different infrastructure, energy demands, and opportunities for sustainable improvements.
- More comprehensive incentive discovery: Expand our tax credit, rebate, and government incentive coverage to help property owners identify programs applicable to their specific buildings and proposed upgrades. This could build off our existing web scraper and parse through the values received.
- Contractor and implementation integration: Connect property owners with contractors, gather quotes, compare installation costs, and track retrofit projects from initial approval through completion.
- Continuous sustainability monitoring: Integrate ongoing utility and building performance data to compare actual energy savings and emissions reductions against the original projections. Agents could identify underperforming upgrades and recommend additional improvements.
- Expanded financial integrations: Move beyond the Nessie sandbox toward real financing workflows, helping property owners manage retrofit budgets, track expenses, and understand the long-term financial performance of their investments. This could also expand into apartment complexes to see overall rent and expenses.
Our ultimate goal is to make sustainability assessments and retrofit planning a standard part of real estate decision-making, giving property owners the information and financial tools they need to make better long-term investments.