Cholo.ai

Turns become sound. Cholo.ai guides blind and low-vision students across campus by ear, then finds them free food, a clinic, or a page it can read aloud.

Comment

Inspiration

Cholo means "let's go" in Bengali.

Most navigation apps assume you can see the screen. Take the screen away and there's not much left. I wanted to see if the whole thing could work by ear instead.

Partway in I found out there's an actual standard for this — Wayfindr's Open Standard, ITU-T F.921 — and reading it showed me I'd built several things wrong. That changed the project more than anything else did.

What it does

Turns become sound. A pulse speeds up as you get closer to a turn, panned left or right depending on which way you're going, then says one instruction at the corner. The same thing can vibrate instead, so you don't need headphones. That matters — headphones in traffic mean you can't hear the car.

Routes avoid stairs properly. The step-free route to the campus pantry is 3,123 feet. With stairs allowed it's 2,317. That detour is the point.

It looks for free food before it spends your money: 1,007 Virginia pantries with hours and phone numbers, plus campus events. VT tags its own events "Meal-based Event," which is better than guessing from the description.

It reads pages with the camera, continuously, so you don't have to press anything. Printed text is read on your phone. Handwriting goes to a vision model.

It finds clinics with real phone numbers and doesn't try to tell you what's wrong with you.

When it buys something, it checks the shop's identity first, agrees a spending limit, then stops and waits for you to say confirm. There's a fake merchant in the demo that fails the check and gets refused out loud.

Everything works by voice. Everything also works with no API key.

How we built it

React, TypeScript, Vite, Tailwind. Groq for the agent, with tool calling — fifteen tools it can chain, so "I'm hungry and I'm broke" becomes check food, find the open pantry, route there, in one go.

The beep is plain Web Audio. One oscillator per pulse, cadence and pitch driven by distance.

Maps are OpenStreetMap through Overpass. I didn't use Google Maps on purpose: it doesn't expose steps, kerbs or surface tags, and those are what step-free routing runs on.

Three real datasets are bundled — Virginia pantries, USDA food access by county, and 231 Forest Service trails pulled out of a 118 MB national file down to 186 KB.

Deployed on Vercel with the API key server-side.

Challenges we ran into

GPS isn't accurate enough for "turn left in 20 feet." It's good to about 5–15 feet outdoors, worse near buildings. So I changed what the beep means: it warns you a turn is coming, and the spoken line tells you what to feel for, like a railing.

My instructions were describing instead of instructing. "The stairs are in front of you" doesn't make anyone move. Every line now has a verb in it.

"Take me to the pantry" contains the word pantry, so it was being read as a food request instead of a navigation one. A test caught that, not the demo.

Crisis handling couldn't go through the model. If someone says something frightening, a rate limit shouldn't change the answer. It's plain code that runs before the agent sees anything. Writing the tests found two phrasings I'd missed.

Campus calendars block browser requests, so events didn't work until I moved the fetch to a server function.

And the first version drew routes straight through buildings, because I'd made the coordinates up.

Accomplishments that we're proud of

The beep actually feels like approaching something, not just getting faster. That took tuning by ear.

179 tests, no framework, one command. They caught five bugs, including one that broke the build entirely.

Nothing breaks when something's missing. No key, no network, no camera, no mic — each one has a fallback and the app tells you which it's using.

The crisis response doesn't touch the LLM. At a hackathon full of LLM wrappers, the most important thing in mine deliberately doesn't use one.

What we learned

The hard part wasn't the code. It was not overclaiming.

Everything I cut — reading emotions off faces, triaging injuries, continuous sign reading — I cut because it sounded good and wouldn't have worked. The person least able to catch a confident mistake is exactly who this is for.

Also: most of this data already existed. The Forest Service records trail grade and surface on every segment in the country. Universities tag their own events as having food. OpenStreetMap maps steps separately from paths. None of it needed inventing.

And a model answering a health question from memory sounds exactly as confident as one reading from MedlinePlus. Only one of those should be near a user who can't check.

What's next for Cholo.ai

Test it with someone who actually needs it. Nobody blind or low-vision has used this yet, and the standard is clear that they should be part of building it. Ten minutes at disability services is worth more than another feature.

Indoor positioning, because GPS stops at the door and that's where it gets hard.

Let students report a broken lift or a new fence by voice — they know days before any dataset does.

More campuses. Most US universities run Localist, so it's mostly a calendar URL and a bounding box.

Built With

Share this project:

Updates

Submission history