EchoPath

Inspiration

A white cane is one of the most honest tools ever designed. It tells you the truth about the ground: the curb, the step, the crack in the sidewalk. What it cannot tell you is the rest of the world.

That is the gap we could not stop thinking about. Not “how do I get across town,” but the three feet between your shoes and the next safe step. For more than 250 million people living with vision impairment, that gap is not a UI problem. It is the difference between leaving the house and staying home. Between independence and waiting for someone else.

What it does

EchoPath is an iPhone-first walking companion for blind and low-vision pedestrians. It does two things at once, and it never lets the second one wait on the first.

LiDAR sees the path. On iPhones, ARKit scene depth samples the world in front of the camera roughly ten times a second. EchoPath does not just say “object detected.” It models three lanes — left, center, and right — filters out the floor, trusts closer hazards first, and pays special attention to obstacles from the ground up to head height. A trash can at your ankle and a branch at your face are not the same kind of danger. The system knows that.

Spatial audio tells you where to go. Instead of shouting “left, left, left,” EchoPath speaks through the body. Distinct HRTF tones pan into the left ear, the right ear, or the center of the head. One cue per second. Learnable. Calm. Understandable even while speech is playing. Straight, left, right, and stop each have their own signature, matched with Core Haptics so the warning still lands if the street is loud.

Navigation becomes orientation, not just a route. MapKit builds a walking route. Core Location tracks you. But a turn instruction is useless if you are facing the wrong way. EchoPath uses compass heading to hear the difference: rotate left, rotate right, and then, when you are aligned with the path, go ahead. Object avoidance stays on the whole time. If a pole appears in your lane, the local decision engine overrides the route. Safety never waits on the cloud.

The phone listens when you cannot look. Say “EchoPath, what’s ahead?” and the live camera plus LiDAR facts go to OpenAI, which answers in one spoken sentence about what is actually in front of you. “Where to?” “Describe.” “Stop session.” Hands stay free. The cane stays in your hand.

The brain of the agent lives on Cloudflare. OpenAI reasoning and ElevenLabs voice run through a Worker, with keys in secrets, not on the device. Immediate obstacle warnings stay on the phone. The cloud is a companion, not a bottleneck between you and a warning.

How we built it

The sensing stack is ARKit + LiDAR scene depth, with a Vision fallback on phones that do not have depth. A three-lane occupancy model, confidence filtering, and a unified decision engine sit between the sensors and the user. Route geometry looks ahead along the actual path polyline, not just the next named turn, so heading alignment matches the sidewalk you are on.

The Cloudflare Worker is the orchestration layer: POST /v1/reason for multimodal scene questions, POST /v1/tts for speech. Sentry watches both the client and the Worker. We left explicit adapter boundaries for future hardware — BLE haptics, Huawei OMNI, QNX — because the iPhone is the brain today, not the whole future.

Challenges we ran into

LiDAR is noisy. Floors look like obstacles. Distant clutter looks urgent. We had to reject the ground plane, require enough agreeing samples, and still respond fast enough that a close hazard is not announced after you have already reached it.

Then there was the freeze. Starting ARKit, MapKit, Core Location, and the microphone at the same moment looked fine in a simulator and locked a real iPhone. We had to sequence startup, isolate camera and transcript views so the whole page did not redraw on every depth frame, and keep the warning path on-device so a cloud round-trip could never be the thing between a person and a pole.

What's next

• Field testing with orientation-and-mobility specialists and blind pedestrians, in controlled environments, with a sighted spotter.
• Tighter LiDAR calibration.
• Stronger AirPods head-tracking.
• Reliable indoor positioning.

Built With

• arkit
• avfoundation
• avspeechsynthesizer
• cloudflareworkers
• core-location
• coremotion
• elevenlabs
• lidar
• mapkit
• openai
• swift
• xcode