NavEasy

NavEasy is a spatial intelligence navigation engine that simulates how people with accessibility needs experience the built world, from the street to the bathroom door.

Comment

Inspiration

More than one in four adults in the US has a disability, according to the CDC. A listing tells them the rent and the number of bedrooms. It doesn't tell them whether a wheelchair fits through the bathroom door, whether there is a step at the entrance, or whether a blind person can walk from the bed to the bathroom without a coffee table in the way. People find out by visiting, one home at a time.

Buildings are already scanned in 3D and photographed from the air. We wanted to use that data to simulate how a person with a specific need would actually experience a space, before they ever go there.

What it does

NavEasy is a spatial intelligence navigation engine. It simulates how people with accessibility needs experience the built world, from the street to the bathroom door, for four needs: wheelchair users, people who are blind or have low vision, people who are deaf or hard of hearing, and older adults.

  • Inside. From each Matterport scan we build a 5 cm floor grid and color it by where a person can go, turn and pass. Doorway widths, thresholds, turning room, stairs and built-in counters are measured from the scan.
  • The building. Elevator or walk-up, age, and accessibility evidence from CoStar records.
  • The street. Routes to a pharmacy, clinic or bus stop, with crossings and curbs from OpenStreetMap and slope measured from Vexcel surface heights.
  • Furniture. Tell it what you own and it places each piece differently for each need, with a reason per piece.
  • Moving cities. Give it a Los Angeles address and it finds Houston neighborhoods where daily places are as close.
  • A voice agent that works like a broker. Press one button and it asks what you need, reads back three homes, tours the one you pick from the front door, and furnishes it. A blind user never needs the screen.
  • The building you are standing in. We loaded the hackathon venue's scans, so the agent can say how to reach the classroom from the entrance and how wide its door is.

How we built it

The core is geometry, not a language model. For every free cell of the floor grid we take the distance to the nearest wall or obstacle, and the local clear width is twice that:

$$ w_{\text{clear}}(x, y) = 2 \cdot \min_{o \in \text{obstacles}} \lVert (x, y) - o \rVert $$

A doorway passes for a wheelchair when its scan width, minus a 5 cm allowance for the frame, is at least 32 inches:

$$ w_{\text{scan}} - 0.05\,\text{m} \;\ge\; 0.815\,\text{m} $$

We use the 2010 ADA Standards as design benchmarks: 32 inch doors, 36 inch walkways and a 60 inch turning circle.

  • Data: CoStar property and listing records, the Matterport Model API (wall graph, 3D mesh, panoramas), Vexcel imagery and surface heights, OpenStreetMap and Valhalla routing.
  • Engine: Python with numpy, scipy and shapely, served by FastAPI.
  • Agent: one entry point, three paths. Voice runs on local open-source models. Typed questions go to a Claude agent with seven tools. Multi-step searches run as a LangChain deepagents plan with four subagents: building (CoStar), inside (Matterport), outside (Vexcel and routes), and a checker that re-runs every failing finding.
  • App: React, MapLibre and three.js, with our heat map drawn inside the Matterport 3D tour.
  • Speech: Whisper and Piper running on the laptop, so nothing a user says leaves the machine.

Challenges we ran into

  • Matterport's object annotations were empty on every scan, so we found furniture by comparing furnished and furniture-removed panoramas, and built-in fixtures from the 3D mesh.
  • About half of the scans have no wall graph. We show Matterport's own floor plan for those and say plainly that there is no heat map.
  • A planned agent run takes about 85 seconds live, too slow for a table demo, so the demo requests are cached and replayed.
  • Keeping the model honest. It never supplies a fact: every number in an answer is matched against the tool results before it is returned.

Accomplishments that we're proud of

  • 13,771 scans indexed, 6,551 measured inside, more than 66,000 doorways, and 4,670 buildings scored across two cities.
  • A blind user can find, tour and furnish a home entirely by voice.
  • The agent shows its plan, each subagent's task and every tool call under the answer.
  • 74 automated tests that run offline in under five seconds.

What we learned

  • A 3D scan holds far more than a virtual tour: the mesh alone gives counters, stairs and level changes.
  • Small local models are good enough to understand a request when they are never asked for facts.
  • Saying "we could not check this" makes the rest of the answer more believable.

What's next for NavEasy

  • Check our door widths against tape measurements at scale.
  • The approach to the door: entrance steps and parking, from Vexcel angled photos.
  • Street-level data for Los Angeles, and the same agent over text message.
  • An accessibility filter on listing sites for every home that has a scan.

Built With

Share this project:

Updates

Submission history