Tandem

Inspiration

Cornell is built on a hill. Google Maps says the walk from Noyes Community Recreation Center to Goldwin Smith Hall takes about 9 minutes, and that's true if you can climb stairs. The route it gives you goes up Libe Slope and includes 14 steps with no ramp. For a wheelchair user, that 9-minute walk is a dead end, and the map never says so.

Map apps answer how long a walk takes. They don't answer whether you can make it. Most people find out at the bottom of the stairs. We wanted a scout that walks the route first, so you don't have to.

What it does

You text Tandem where you're going, like "Noyes to Goldwin Smith, I use a wheelchair." About a minute later it texts back:

Every candidate route, checked photo by photo with Street View and Claude vision, cross-checked against OpenStreetMap (stairs, curbs) and elevation data (steep grades)
The route you can actually take, compared with Google's: on our demo route, Tandem's pick avoids all 14 steps at about the same walking time
Proof: each problem comes with its photo, a red box around it and the photo's date
The entrance: which door of the destination is step-free, or an honest "the map doesn't say"
Weather and the bus: if walking looks bad (ice, snow, stairs on every route), a TCAT bus option
A flythrough GIF of the walk and a trip page with every photo

How you get around changes what it looks for: wheelchair (steps, missing curb ramps, steep grades), stroller (the same barriers, weighted lower), or walking alone at night (street lights, isolated paths, blue-light phones, places open late). After a trip, you can report what it missed, and that report becomes a flag on the next person's scout and on our campus barrier map.

No app. No download. Just a text.

How we built it

Messaging: an iMessage bot on Photon (Spectrum), with a Claude agent handling the conversation and SQLite remembering profiles, trips and reports
Routes: the Google Routes API in walking mode, with alternative routes
The scout: we sample points along each route and fetch Street View frames facing the direction of travel, plus angled views at corners. Claude vision checks each frame for barriers and takes a second look at anything uncertain
Cross-checks: OpenStreetMap through Overpass (stairs, curbs, entrances, lighting) and elevation from OpenTopoData with USGS as a fallback, to compute grades:

$$ \text{grade} = \frac{\Delta h}{\Delta d} \times 100\% $$

Scoring: each flag gets a severity from 1 to 3, squared so one real barrier outweighs several small ones:

$$ \text{score}(r) = \sum_{f \in \text{flags}(r)} \text{severity}(f)^2 $$

The route with the lowest score wins, with walking time as a tiebreaker.

Visuals: a live "mission control" dashboard (Leaflet + OpenStreetMap, streamed with server-sent events) where you watch photos get checked in real time, plus annotated photos, the flythrough GIF, trip pages and a campus barrier map
Stack: Bun and TypeScript. Our three folders (messaging, scout, visuals) shared one typed contract and a demo fixture, so we could build in parallel from the first hour

Challenges we ran into

Street View doesn't see everything. Campus paths often have only 2009 imagery or none at all, and the best photos of Libe Slope were user uploads the API can't reach. So we stopped relying on vision alone: stairs come from OpenStreetMap, grades from elevation data, and Tandem tells you which stretches it couldn't check instead of pretending.
Our photos didn't match Google's route. We picked "obvious" problem spots by eye, and then the scout showed that our staircase was 54 m off Google's actual walking route. We added a check that every flag sits on the route, and rebuilt the demo around the stairs that are really on it.
APIs we couldn't enable. When Google's static map and elevation APIs weren't available on our key, we switched to OpenStreetMap tiles and public elevation services without changing any function signatures.
One number per user. Photon gives each user their own number, so judges can't all text one line. We built a demo replay and shareable trip pages so anyone can watch a real scout.
Small bugs with big effects: Windows file paths broke every dashboard photo, and a "90%" confidence label read as a 90% slope.

What we learned

Honesty is a feature. For accessibility, a confident wrong answer is worse than none. Showing photo dates, admitting unchecked stretches and saying "the map doesn't know which door" made Tandem more trustworthy, not less.
Cross-checking beats any single source. Vision, OpenStreetMap and elevation data each have gaps, but they rarely have the same ones.
A shared contract makes speed possible. Agreeing on the data types and a fake scout result in hour one let three people build three parts at once and merge them cleanly.

What's next

Re-checking your route right before you leave and texting only if something changed, live elevator outage alerts, a low-vision mode, and step-free entrance info for hotels and venues.