WanderLens

Autonomous urban explorer & dual-camera photography scout, built end-to-end on Google's stack

Comment

Autonomous urban explorer & dual-camera photography scout, built end-to-end on Google's stack: the Gemini API, the Agent Development Kit (ADK), Google Places API, the Google Weather API, Google Cloud Text-to-Speech, and a Google Cloud Run deployment with a Google Cloud Build/Workload Identity Federation pipeline behind it.

Live demo (no setup needed): https://wanderlens-537942792610.us-central1.run.app (open it on your phone).

Built with Google

Every piece of intelligence and every piece of infrastructure in this project is Google's, which felt like the right way to build something for a Google hackathon:

  • Gemini API: powers all three reasoning agents (Scout, Generator, Critic).
  • Google Agent Development Kit (ADK): the SequentialAgent and LoopAgent primitives that orchestrate them.
  • Google Places API (New): live landmark detection and on-demand vantage-point search.
  • Google Weather API: live temperature, cloud cover, and daylight for the Scout's environment read, using the same key already configured for Places.
  • Google Cloud Text-to-Speech: the actual spoken narration the traveler hears.
  • Google Cloud Run: the single-container production deployment at the link above.
  • Google Cloud Build + Workload Identity Federation: the GitHub Actions pipeline that redeploys automatically, authenticated to Google Cloud with no stored keys.

Inspiration

I take a lot of photos when I travel, and I kept running into the same three annoyances. Audio guides are static: they don't know or care that it's overcast right now instead of the sunny day the recording assumed. Figuring out the right shutter speed or ISO for a dim, backlit cathedral interior is genuinely fiddly, and I'd either underexpose the shot or end up with motion blur from a too-slow handheld speed. And when I tried just asking an LLM for "a good spot to photograph St. Paul's from," it would confidently suggest gardens and viewpoints that don't exist.

That last one is what actually pushed me toward an agentic build rather than a single prompt. If a model is going to suggest a location, it should be grounded in a real API result, not free-associated. That constraint shaped a lot of the architecture.

What it does

WanderLens watches your GPS as you walk. When you get near a landmark, it checks the weather, calculates the sun's position, and pulls up any personal photography notes relevant to the scene. Then it writes a short spoken script and works out the exact camera settings for your gear. Before reading any of that back to you, a second agent checks the first agent's math and sends it back for a rewrite if the shutter speed it picked would cause camera shake, or if the script runs too long to read aloud in a reasonable time.

You configure up to two cameras or phones once (whatever you actually own), and every recommendation is tailored to that gear specifically, not a generic "set ISO to 800" answer.

How I built it

It's a Python/FastAPI backend and an Angular PWA frontend, with four separate agent contexts orchestrated by Google's Agent Development Kit and powered by Gemini (see the architecture diagram in the gallery above: GPS ping → Scout Agent → RAG Retriever → Generator/Critic ADK LoopAgent → Cloud Text-to-Speech delivery).

A few decisions worth calling out:

  • The Generator/Critic loop is ADK's LoopAgent primitive, not a hand-rolled while-loop with string parsing. It comes with a bounded iteration count and a defined fallback if the Critic never approves, so it can't spin forever.
  • RAG runs embedded, not as a separate service. ChromaDB's Python client supports true in-process mode, and I wrote a small offline hashing embedding function instead of the default one (which downloads an ONNX model on first use, not something I wanted to depend on with unreliable hackathon wifi).
  • Every external call has a fallback. No Gemini key → the whole pipeline runs on deterministic mock logic instead (and still makes a real weather call and a real RAG query, so the behavior you see in mock mode is close to the real thing). No Places key → a curated list of six hand-verified London landmarks. No TTS → a placeholder tone instead of a broken response. The idea was that a flaky demo network or an expired key shouldn't take the whole thing down mid-judging.
  • Vantage-point suggestions are opt-in. Early on this was automatic, but a live Google Places call is billed per request, so I moved it behind a "show me a nearby vantage point" button instead of firing it on every landmark match.
  • Deployment and CI/CD are Google Cloud end to end. A two-stage Dockerfile builds the Angular PWA and the FastAPI backend into one container, deployed to Google Cloud Run via Google Cloud Build. A GitHub Actions workflow redeploys automatically on every push, authenticated to Google Cloud through Workload Identity Federation rather than a stored service-account key.

Challenges I ran into

The one that cost the most time: ADK's LlmAgent can be given a structured output_schema (so the Critic's verdict comes back as clean JSON) or a loop-control tool like exit_loop, but not reliably both in the same turn: the model can't finish a schema-constrained response and also emit a tool call in one pass. I found this out by watching the loop just... never exit. The fix was pulling loop control out of the LLM entirely: the Critic still returns a structured verdict, and a separate CriticGate (a plain Python BaseAgent, no model call involved) reads that verdict from session state and decides deterministically whether to exit the loop or retry the Generator with feedback attached.

The other recurring one was just keeping up with model deprecations. Gemini 1.5 was retired partway through working on this, so I stopped hardcoding a specific model ID and pointed the config at the -latest aliases instead, with the actual model name pulled from an env var so it's a one-line change whenever Google ships the next generation.

I also hit an occasional bug I never fully closed out: the live Scout agent sometimes calls find_nearby_landmarks with its arguments scrambled: the search radius shows up in the longitude slot. I traced it with ADK's event log and confirmed it's a real (if rare) tool-calling quirk rather than a bug in my own code, but I didn't have time to harden against it before the deadline. It doesn't affect the mock pipeline or the test suite, since neither depends on live tool-calling.

Accomplishments I'm proud of

  • The Critic genuinely catches things. I can point it at a low-light scene and watch it reject a first draft for a shutter speed that would blur, then approve a corrected second draft. That's the actual "self-reflection" moment the whole project is built around, not just a demo script.
  • You can clone the repo and run the entire stack (backend, frontend, all 16 backend tests) with zero API keys and zero cost. I wanted a judge (or anyone) to be able to see real behavior without needing to hand over a credit card first.
  • It's a real, installable PWA, not just a local dev server: offline-capable shell, a proper mobile layout, real Cloud TTS narration, working Google Maps links to the suggested landmark, and a live Cloud Run deployment.
  • 49 automated tests across the stack (16 backend pytest, 25 frontend unit, 8 Playwright end-to-end against a real browser), enough that I could keep refactoring the UI and the reflection loop right up to the deadline without breaking things I couldn't see.

What I learned

Splitting an agent's responsibilities into isolated contexts (Scout only sees GPS + tool results, the Generator only sees the Scout's output + RAG notes, the Critic only sees the draft) made the system dramatically easier to debug than I expected. When something went wrong, I only ever had to reason about one agent's inputs and outputs at a time, instead of untangling one long context window. I also came away with a healthier respect for how much of "agentic AI" quality work is actually about designing the boundaries and fallback paths around the model, not the prompt itself: the deterministic CriticGate and the mock pipeline both mattered more to how well this ended up working than any single prompt did.

What's next

  • A vision-based Critic: let the traveler snap the actual photo they took, and have Gemini check composition and exposure against what it recommended, instead of only reasoning about settings ahead of time.
  • Spatial audio cues that nudge the traveler to physically turn toward a specific architectural feature.
  • Direct RAG ingestion from an Obsidian vault or Lightroom catalog, so the notes it draws on stay in sync with how my own shooting style actually evolves.

Try it yourself

# Fastest: just open it, no setup
https://wanderlens-537942792610.us-central1.run.app

# Or run it locally
./start-all.sh   # backend on :8000, frontend on :4200
./kill-all.sh    # stop both

# Tests
cd backend && pytest              # 16 tests, no API keys needed
cd frontend && npm test           # 25 unit tests
cd frontend && npm run e2e        # 8 end-to-end tests (Playwright)

Built With

Share this project:

Updates

Submission history