Inspiration
GPS stops at the front door. Inside, signs are written for people who can see them and routes assume you can take the stairs. InView turns a building into a map you can hear, follow and roll through, so blind and disabled visitors can find their own way.
What it does
Scan a building once with a LiDAR iPhone, then get step-free routes that avoid stairs, narrow doors and unsafe ramps, guided by AR arrows, a map and voice. Signs are read aloud, and the nearest exit is one tap away.
How we built it
- An iOS app on RoomPlan and ARKit LiDAR for scanning, relocalization and AR guidance.
- A vision model proposes elevators, ramps and stairs; LiDAR depth confirms each one.
- On-device OCR reads signs, which LiDAR places in 3D and binds to the nearest door.
- Routing and geometry live in a tested Swift package (
AccessCore); a Rust server (hosted on Vultr) handles accounts and map sharing. - ElevenLabs for voice, Gemini for voice questions, Mapbox for the outdoor map.
Challenges we ran into
- Models misread or hallucinate, so every AI detection has to be confirmed by measured depth.
- Placing signs and obstacles accurately in 3D from a handheld phone.
- Recognizing a space from the saved scan with no QR codes or beacons.
Accomplishments that we're proud of
- A reconstructed Floor 1 with measured, elevation-aware routes.
- Step-free routing based on measured door widths and ramp slopes, not assumptions.
- Works with VoiceOver, so one map serves wheelchair users and blind visitors.
What we learned
- AI is good at proposing and bad at certifying; pair it with a physical measurement.
- Hardware is more advanced and faster than we estimated.
What's next for InView
- Between-floor directions from published building connections.
- AR guidance on shared maps, not just on the phone that scanned.
- More real-device testing of multi-room scans and relocalization.
Log in or sign up for Devpost to join the conversation.