Inspiration

A friend’s mom is blind. When they ride the train together, he describes the scenery outside—the landmarks, the neighborhoods, the little things that make the trip more than a ride. But he doesn’t go to SBU, and he can’t always come back to share those moments with her.

That got us thinking: what if the view could tell its own story?

We started Gaze with her in mind, then realized how many of us miss the world outside for a different reason: we’re looking down. In a world of endless scrolling, we wanted to make looking out the window interesting again.

What it does

Gaze turns your train ride into a narrated journey through the places around you. It follows your route and tells you about nearby landmarks, local history, and stations as you travel.

Curious about something outside? Point your phone toward it to discover nearby places, hear their stories, and save your favorites. Or start the audio tour and let the journey come to you.

How we built it

We built Gaze as a Next.js progressive web app, using GPS and phone orientation to connect where you are with where you’re pointing.

MTA route data and OpenStreetMap provide the geographic foundation, while Supabase and PostGIS power our place catalog and nearby searches. Gemini generates stories from retrieved sources, an evidence-checking pipeline filters unsupported claims, and ElevenLabs brings the narration to life.

Challenges we ran into

A moving train makes “what am I pointing at?” surprisingly complicated. Position, heading, phone tilt, and nearby landmarks all have to line up, and proximity alone doesn’t tell us what’s visible through the window.

We also had to make the audio feel like a thoughtful companion. Stories needed to queue without overlapping, respond to discoveries, and stay quiet when there wasn’t anything worthwhile to say.

Then there was trust: an interesting story is only useful if we have evidence for it. We built source checks and fallbacks instead of letting the model fill gaps with confident guesses.

Accomplishments that we're proud of

We connected route tracking, directional discovery, sourced storytelling, and narration into one working experience.

We’re especially proud that riders can explore actively by pointing or simply listen as they travel. That gives Gaze a purpose beyond identifying landmarks: it offers another way to experience the landscape.

What we learned

We learned that building an audio experience means designing timing and silence as carefully as the words themselves.

We also learned that an accessibility-inspired idea can reveal a broader need. The same experience that could help someone hear the view can help someone else finally look up from their feed.

What's next for Gaze

Our first priority is testing with blind and low-vision riders, including the person who inspired the project. We want their feedback to shape the controls, narration, and descriptions before we call Gaze accessible.

From there, we want to expand route coverage, improve landmark matching, and make narration more reliable with weak connectivity.

There’s a whole world between your stops. We want to help you notice it.