Inspiration
Opening a small business means finding a space, and that usually takes months. Owners compare listings, go on tours, pay people to research zoning and build-out costs, and pass details between brokers, contractors and designers. Expensive dealbreakers often show up only when a lease is nearly signed: a door too narrow for a wheelchair, no sink in the exam room, a kitchen with no way to vent a hood through the roof, or a storefront on a block that's empty at lunch. And when owners finally find a space worth pursuing, their first message to the broker is often just "is this available?", which tells the landlord nothing about why they'd be a good tenant.
Hermit crabs don't build their shells. They find one that already fits. We wanted small business owners to work the same way: describe what you need, find the space that's already built for it, and know exactly what's left to do before you sign.
What it does
Hermit is an AI agent that screens real spaces by checking all three layers of the same address: the CoStar record, the Matterport scan of the inside, and the Vexcel view from above. It finds what fits and what's left to do. It's a screening tool, not an inspection.
- Tell Hermit what you need: by chat or voice, or by sketching a room layout. Hermit turns that into an editable spec of rooms, sizes, adjacency and must-haves.
- Watch the agent work: Hermit plans a search over about 15,000 scanned spaces in Houston and Los Angeles. A crew of specialists each read one layer, and every step is shown live: what it checked, what it dropped and why, and where it looked closer.
- Shell Reports: for each of up to 5 matches you get:
- a floor-plan overlay of your rooms on the real measured rooms
- a cross-check of each important claim against every layer
- flags for zoning, wheelchair access, parking, fire and power
- a foot traffic heatmap of the block by time of day
- the remaining build-out work
- questions for the landlord
- Step inside and change it: walk the real Matterport tour and tell Hermit "furnish this room as an office" or "merge these two offices". It places measured furniture inside the walls and off built-in counters, and re-scores the fit after each wall change.
- Talk to your broker: landlords screen tenants much like employers screen candidates. Hermit builds a tenant packet, a one-page resume for your business: your exact proposed use, move-in date and lease length, your finances and experience, your furnished floor plan, the work left to build, and open questions. It drafts the message and opens it in Gmail. You always press send yourself.
Who switches it on Monday: the owner looking for their first or next location, and the tenant-rep broker who today does this screening by hand for every client.
How the layers work together
Each layer alone is incomplete. CoStar knows the building but not the rooms. Matterport knows the rooms but not the zoning or the roof. Vexcel sees the roof and the parking lot but not inside. Hermit cross-checks each claim that matters against all three:
| Claim | CoStar (record) | Matterport (inside) | Vexcel (above) |
|---|---|---|---|
| Built for your use | property type / subtype | room and object labels from the scan | — |
| Size | listed square footage | sum of measured room areas | building footprint |
| Restroom in the space | — | restroom found in the room graph | — |
| Step-free route | floors, elevator | door widths along the route | — |
| Parking | parking ratio on record | — | parking lot visible from above |
| Kitchen-ready | sprinklers | hood / kitchen in the scan | rooftop exhaust |
A claim is confirmed only when two independent layers agree. When they disagree, for example when the listed square footage is much larger than the measured rooms add up to, Hermit shows the conflict instead of averaging it away and turns it into a question for the landlord. A claim with no evidence is unknown, never pass.
Agentic depth
Hermit is an agent that uses tools within a budget, not a chat wrapper. Gemini plans; code measures.
- Tools the agent chooses between:
search_shells,screen_sites,evaluate_candidates,match_layout,inspect_view(a targeted look at one scan, e.g. "is there a sink in this room?"),research_regulations,consult_specialist,drop_candidateandsubmit_reports. - A narrowing funnel, so the expensive spatial work only goes to plausible matches: $$50 \rightarrow 20 \rightarrow 8 \rightarrow \le 5 \text{ reports}$$ Hard filters cut wrong city, size and type. Cheap site, parking and zoning checks screen the next 20. Only the top 8 get full room matching. (Houston has no zoning, so Hermit flags deed restrictions there instead.)
- Specialists with veto power: Inspector walks the scan, Surveyor reads from above, Neighborhood scout reads the area and its foot traffic, Code checks city, state and federal rules, Access checks the accessible route, Systems checks power and AC, Estimator prices the build-out, Advocate handles the landlord, and Skeptic argues the other side. A veto is binding. If it removes the leading candidate, Hermit drops it, says why, and moves on to the next one.
- It changes course on evidence: drops always come with a written reason, and the agent looks closer when an uncertain fact could change the ranking. The trace shows real drops and reranks, not a script.
- Independent auditor: before reports ship, a separate verifier rechecks every must-have and rejects any pass/fail claim that has no source.
- After the search, the same agent keeps acting: one Hermit conversation, from Find a space through the report page, sees what's on your screen. It routes each request to the right tool: answer a question, edit walls through typed fit-out operations that code applies and rescores, furnish the 3D tour, or draft the broker packet.
Spatial and visual understanding
- Room graphs from scans: every scan becomes a graph of measured rooms, doors and adjacencies. Your requested rooms are matched to measured rooms by area and adjacency. Scan coordinates have no fixed unit, so we recover the scale: $$\text{ft per unit} = \operatorname{median}_r \sqrt{\frac{A_r^{\text{measured}}}{A_r^{\text{polygon}}}}$$
- Wheelchair routing: a route search runs through the measured doors. A hinged door's clear width is $$w_{\text{clear}} \approx w_{\text{opening}} - 2 \text{ in}$$ and a room counts as reachable only if every door on the route has $$w_{\text{clear}} \ge w_{\text{chair}}$$
- Furniture in the live 3D tour: Hermit finds the visible floor, sizes models to real furniture dimensions, keeps pieces inside the walls and off built-in counters, and tells you what it couldn't place and why.
- From above: Vexcel imagery gives the footprint, parking lot and rooftop equipment for each match.
- Foot traffic heatmap: each report maps how busy the block around the space is, on a street map you can switch between morning, midday, evening and night. Hermit combines several sources:
- hourly busyness of nearby venues
- road traffic counts on the nearest major road
- how many people work within a 10-minute walk (the weekday lunch crowd)
- transit stops and places to eat nearby
- Census income and age for the block group
A café owner sees whether the street is busy at 8 AM. A clinic sees whether patients can park and get there by transit. These are labeled proxies, never counts of people walking past the door.
How we built it
- Data: CoStar records, Matterport scans and Vexcel imagery, preprocessed into a private local bundle, so search is fast, cheap and works offline. The neighborhood layer adds:
- BestTime venue busyness
- state DOT traffic counts
- US Census LEHD jobs and ACS demographics
- OpenStreetMap transit stops and businesses
All of it is prefetched per site. A recorded replay keeps the demo running if the venue Wi-Fi doesn't.
- Agent: Gemini orchestrates over typed tools. Geometry, compliance and cost always come from deterministic code.
- Frontend: Next.js and React, the Matterport SDK for furnishing inside the tour, an SVG floor-plan overlay, and a map-tile heatmap for foot traffic.
- Citations: every number or claim in a report links to its source (a CoStar field, a scan room, a Vexcel image, a neighborhood dataset or a rule row).
Challenges
- The data didn't match its labels. The "commercial" scans were mostly apartment units. We checked classifications ourselves and are honest about which spaces a scan actually verifies.
- Geometry isn't proof. A scan shows a room is 120 sf, not that the wiring behind the wall meets code. Keeping measured facts, cross-checked facts, estimates and unknowns separate shaped the whole report.
- Foot traffic data doesn't exist for most addresses. No dataset counts people walking past a given door. We built the heatmap from proxies (venue busyness, road counts, nearby workers), labeled it as an estimate, and show a typical weekday pattern when a site has no busyness data.
- Putting furniture into a live 3D tour meant finding the floor, fitting real dimensions, and queuing camera moves, because moving too early blacked out the tour.
- One agent that does everything. Answering, editing walls and furnishing each use different tools, and routing a message to the right one took several iterations.
What we learned
- Let the model plan and let code measure: the LLM is great at intent, but numbers and compliance results have to be deterministic and cited.
- The layers are most valuable where they disagree. A conflict between the record and the scan is exactly the question a tenant should ask before signing.
- "We don't know, ask the landlord this" builds more trust than a confident guess.
- Finding the space is only half the job: a first message that answers the landlord's screening questions makes a far stronger case than "is this still available?"
Built With
- costar
- gemini
- matterport
- mcp
- next.js
- node.js
- postgresql
- react
- sketchfab
- supabase
- typescript
- vercel
- vexcel