Waymo Haven

Inspiration

When visiting cities, we often feel lost in them with no clear direction on how to make the most of your stay here and we wanted to answer this for tourists. Waymo just launched robotaxis in Miami, but a self-driving ride is still just point A to point B — you stare at a phone while the city goes by. We wanted the empty seat time in an autonomous vehicle to become the destination itself: a car that already knows the roads should be able to show you the best parts of them, tailored to your mood, not just the fastest ones.

What it does

Waymo Haven allows users to have a fully personalized and automated guided tour experience. Users can select their desired mood and preference on what they want to see, as well as a preferred time frame. Waymo Haven will create a catered guided tour based off the users current mood in the vehicle with the help of Gemini and ElevenLabs to voice over to cover the significance of each location. For example, if you were in the mood to visit murals, Waymo will pick the top locations based on the rating given for each street-view from Gemini.

How we built it

Cheapest-insertion over real Dijkstra shortest-path routing — so the path Waymo Haven picks is a real, drivable road, not a straight line. For each stop, we pull the nearest highly-rated business (Google Places) and Wikipedia fact, feed that into an LLM to write a short script, then generate multilingual voiceover (English, Spanish, Haitian Creole, Portuguese) with ElevenLabs. The FastAPI backend serves tours (with Mongo-backed caching so the same mood/time/start combo doesn't get rebuilt), and a Next.js/React map frontend renders the live route, stops, and narration player.

Challenges we ran into

Cost and API keys: Street View and LLM scoring cost real money per call, so we built a fake scoring mode to unblock frontend work before any keys existed, and made every pipeline stage resumable so a Ctrl+C didn't cost us progress.
Trusting the AI's scores: an LLM rating a street photo can hallucinate a mural that isn't there, so we added a verification layer that cross-checks the AI's score against a real Google Places rating for the identified subject.
Tracing a real service area accurately from a static map image into a geo-referenced polygon.
Multilingual voice: not every ElevenLabs model handles every language equally well (Haitian Creole was rougher than the others), so we had to test and pick voices/models per language rather than assuming one model covers everyone.
Timing: narration generates in the background per stop while the user is already looking at the route, so the frontend had to poll and gracefully show partial tours instead of blocking on all the audio finishing first.

Accomplishments that we're proud of

A real end-to-end pipeline: from a traced map image to a scored street network to a drivable, time-budgeted, narrated tour — no step is faked in the final product.
A verification step that catches AI mistakes by checking them against real-world data (Places ratings, Wikipedia) instead of trusting the model blindly.
Fully multilingual narration, not bolted on after the fact — language is a first-class parameter of every tour.
We got it actually deployed and demoable (DigitalOcean + Vercel), with a "bake" step that pre-renders the demo tour so the stage demo never depends on a live API call.

What we learned

As a team, we learned the importance of compromise, and problem solving. But what really helped us complete this project was communicating. There were times where progress wasn't being made which halted others progress. However, we were able to communicate and work together to solve our problems, and ultimately complete our goal

What's next for Waymo Haven

While currently in the prototype phase, we would love to expand this technology at Waymo. We could probably integrate this feature as a unique tourism feature for passengers in a city that will offer a more in depth museum-like look into the city. We believe this technology will expand Waymo's role as a trusted driver in the field of tourism to give passengers a unique experience.