Gnarly

Walk a building once. Watch the floor plan draw itself.
True-color 3D replica of Baker, built from your own photos.
Explore every floor in 3D, then walk it in first person.
Follow a glowing AR path on the real floor.
LiDAR night-vision: see what the phone sees, even in the dark.
LiDAR off vs on: the depth that drives obstacle haptics.
Mapped and demoed on real Cornell buildings: Baker and Bethe.
Multi-zone, multi-floor routes with elevator handoffs.

Inspiration

GPS dies at the front door. Step inside a lecture hall, a dorm, or a hospital and the blue dot just drifts while you guess. The usual fixes are Bluetooth beacons, Wi-Fi surveys, and CAD floor plans: expensive to install, slow to keep current, and out of reach for most buildings. The people who pay the price are the ones who can least afford a wrong turn: newcomers, people with low vision, and anyone who needs a step-free route.

So we asked a bold question: what if the LiDAR sensor already sitting in a Pro iPhone could replace all of that infrastructure? No beacons. No floor plans. No hardware on the walls. Just walk a building once, and anyone can navigate it in AR. Gnarly is our answer: indoor navigation, reimagined.

What it does

Gnarly is a full indoor navigation platform we built from scratch: three connected apps that run off a single walkthrough. Walk a building once with a LiDAR iPhone and Gnarly turns that one pass into a living 3D model, a routing graph, and the map your phone uses to find itself.

Mapper (iOS):

Our mapper watches a floor plan draw itself in real time as you walk, capturing walls, doors, openings, windows, floors, and furniture.
Our own color-baking engine paints the real photos you took onto every surface, so you get a true-color 3D replica of the room, not a gray wireframe.
It anchors each zone with an ARKit world map, generates a walkable route graph automatically, and publishes the whole building to the cloud.
Zones link together through elevators (between floors) and continuations (same floor), so any building, however big, can be stitched together from many scans.

Navigator (iOS):

Search for a destination, or tap start and end points directly in a real-color 3D model of the room.
Look around for a few seconds and our relocalization pins down exactly where you're standing.
Follow a glowing AR path laid on the real floor, with a turn arrow whenever the path leaves the screen.
Our wall-aware router never sends you through a wall, and it carries routes across zones and entire floors.
A real-color minimap and route planner show distance, walking time, and zone changes before you take a step.

Web companion:

Find buildings on a map and explore each one in 3D with its floors stacked.
"Drop to walk": step into any floor in first person, passing only through doors and openings that were actually scanned.
A waypoint editor lets building managers shape the routes, checking every edit against walls and doorways with conflict-safe saves.

We demoed it live on scans of real Cornell buildings.

Accessibility

Accessibility isn't a bolt-on for Gnarly. The same LiDAR that maps the building also guards your path.

Feel obstacles before you reach them: While you navigate, LiDAR sweeps a corridor about 0.8 m wide and 3.5 m ahead along your route. Anything from about 25 cm to 1.75 m high registers as an obstacle, and the phone's vibration ramps with proximity: silent beyond about 3.2 m and full strength within about 0.45 m. A person, a cart, or an open door makes itself felt without you ever looking at the screen.
Works in the dark: LiDAR is an active sensor, so obstacle sensing keeps working where camera-only AR goes blind.
Night-vision view: 1,200 rays per depth frame build a live point cloud of up to 300,000 points, with a sonar-style pulse every 1.6 s. It shows exactly what the phone "sees," driven by the same depth data as the haptics.
Step-free routing: Multi-floor routes go through elevators, with each ride weighted by the number of floors.
Plan before you go: Preview the full route in the app, or walk the building in 3D on the website before you ever arrive.
Landmarks you recognize: Real colors mean you can navigate by "the blue wall by the windows," not abstract polygons.

How we built it

The core idea: one scan, three jobs. The same capture is the visual model, the navigation graph, and the localization map. Keeping a building up to date is as simple as rescanning one zone.

Custom color baking (Swift, RoomPlan, ARKit): RoomPlan gives geometry but no color, so we built our own texturing pipeline. The Mapper saves a keyframe every time the camera moves 25 cm or turns about 11 degrees. Every surface is projected through those camera poses, weighted by viewing angle and distance, and checked against LiDAR depth so occluded surfaces never get painted with the wrong pixels. The result is packed into a JPEG texture atlas consumed by both the website and the Unity minimap.
Wall-aware A* pathfinding: Any edge that crosses a wall without a captured door or opening is rejected outright. Zones merge into one unified graph, with weighted elevator edges carrying routes between floors.
Multi-zone relocalization: Each zone has its own ARKit world map and coordinate system. Cross-zone routes are split into legs, with a handoff from one map to the next.
Navigator: Unity 6 and AR Foundation, with native Swift bridges for Core Haptics, a RealityKit room viewer, and secure session storage.
Web: React 19, TypeScript, React Three Fiber, Vite, and a Node server.
Backend: Firebase Auth, Firestore, and versioned Storage packages, locked down with admin-only writes.
One shared format: Swift, Unity C#, and React all read and write the same versioned package format, so three codebases stay in lockstep.

We built Gnarly with Cursor and Codex. Grok Bot produced our promo material: the demo video, trailer, logo, and this write-up.

Challenges we ran into

Stitching zones together: ARKit tracking drifts over long distances, so a big building can't be one giant scan. We had to break it into zones and relocalize at each handoff, and make those handoffs work seamlessly in both the Navigator app and the website.
Pathfinding in the real world: Deciding where nodes should go and what is truly walkable versus not was harder than it sounds. A graph that looks right on paper can happily route you through a wall, so we made the router wall-aware and gated every crossing on a real scanned door or opening.
No color from RoomPlan: RoomPlan's models are geometry-only, so we engineered our own photo-projection and texture-atlas pipeline from scratch.
Depth-to-screen alignment: LiDAR depth arrives in the sensor's native orientation. Lining it up with a portrait screen took ARKit's display matrix plus careful handling of the aspect-fill crop.
Keeping haptics alive: Core Haptics can silently stop after an interruption, so we re-send intensity several times a second and rebuild the engine when needed.
Three codebases, one team: Swift, Unity C#, and React had to agree on one versioned format, and some of us were developing on Windows, where Swift can't even compile.

Accomplishments that we're proud of

End to end, zero infrastructure: Scan a room, publish it, and navigate it in AR from a different phone, with nothing installed in the building.
Real color on RoomPlan models: Photographed surfaces on both web and mobile, powered by a depth-aware baking pipeline we wrote ourselves.
You can feel the path ahead: LiDAR-driven haptics warn you about obstacles, even in the dark.
Routing that respects reality: Wall-aware paths that carry you across zones and up and down floors.
A walkable building in the browser: Step into a real scanned building and explore it in first person.
Three apps, one format: Three platforms and three languages, all speaking the same versioned package.

What we learned

RoomPlan delivers clean geometry but no appearance. Depth-based occlusion is what puts color in the right place.
Sensor data doesn't line up with the screen by default. Orientation and cropping have to be handled explicitly.
Large-scale indoor navigation has to be designed as a chain of handoffs between coordinate systems, not one perfect map.
A shared, versioned data format is what let three codebases move fast without breaking each other.
RoomPlan's stair detection wasn't reliable enough to trust, so explicit elevator links gave us routes we could count on.