Waypoint: LiDAR-mapped indoor navigation

Inspiration

GPS stops working the moment you walk through a door. For someone who is visually impaired, an unfamiliar building (a hospital, a campus hall, a conference venue) can mean asking strangers for help or memorizing a route in advance. We wanted a building to be able to guide people through it: a sighted volunteer walks it once, and everyone after can find their way independently.

What it does

Waypoint is an iPhone app (for Pro iPhones with LiDAR) with two roles:

Volunteer (sighted): walks a building while the phone scans it with LiDAR. The scan becomes a shared map with corridors, doors, stairs and room labels pinned on it. Maps are shared through the cloud, and anyone can extend an existing building's map instead of re-scanning it.
Navigator (visually impaired user): picks a destination and is guided by spatial audio (the next turn is heard from the direction you need to go), haptics and a clear spoken voice. The phone uses LiDAR depth to notice obstacles, suggests the way around them, and warns about drop-offs such as stairs down and ledges.

We also built a web viewer that shows the scanned building in 3D in a browser, with routes and a walkthrough, so a map can be checked without a phone.

How we built it

Swift and SwiftUI with ARKit and LiDAR to scan buildings and recognize where the phone is inside a saved map, plus Apple's speech, spatial audio and haptics for guidance.
The navigation brain: A* routing to find the best path, bearings to work out which way to turn, and a feedback planner that decides what to say, play or vibrate and when, so cues never pile up on top of each other.
Supabase to store and share one map per building.
A browser viewer (JavaScript and WebGL) that renders the same maps in 3D.
Optional Gemini to word the opening route briefing naturally, with a built-in fallback so guidance never depends on the network.

Challenges we ran into

Aligning scans: extending a building means fitting a new scan into an old one. We added manual alignment from two confirmed locations so it doesn't depend on the phone recognizing the space perfectly.
Not overwhelming the listener: audio guidance must never talk over itself or drop a safety warning. We built a prioritized speech queue where warnings come first and nothing is silently lost.
Big 3D scans: real scans are tens of megabytes, so we limited export size, checked downloads for corruption, and made the viewer load them progressively.

Accomplishments that we're proud of

Turning a walk into a map: a volunteer can walk a real building once and end up with a map of its corridors, doors and rooms.
Guidance you can follow by ear and touch: directions come from the direction of the next turn, with vibration and short spoken instructions, so you never need to look at the screen.
Maps that grow: buildings are shared, and anyone can add new areas to an existing map.
A 3D viewer that lets anyone see and check a building's map from a web browser.

What we learned

Accessibility is mostly about restraint: the right single sentence at the right moment beats a stream of information. We also learned how much faster a team moves when the risky parts, such as routing and speech rules, are built so they can be tested without hardware.