Yatra

Historically-grounded AI walkthroughs - RAG-backed, not hallucinated. Pick any place and era, get a narrated journey through daily life there, with generated scenes.

Comment

Yatra - A Journey Through Time

Yatra is an AI-generated historical walkthrough app. A user picks a place and an era; the backend retrieves grounded historical context via RAG, generates a 3–5 stop narrated walkthrough with images, and serves it as an interactive, shareable experience.

Inspiration

The idea originated in 10th grade when it was formed through two independent concepts. One of my friends mentioned the need for something to be created with the help of which you can restore artifacts. And later back in December 2025 , I had been to The Museum of the Future in Dubai, where I saw a scanner that could display information about the artifact pointed out by the user. Although none of the ideas alone resonated, their synthesis remained in my head. And taking into account the richness of historical places in India, the idea of accessibility of history continued to appear in my mind.

Historical textbooks or Wikipedia provide you with the factual information regarding the certain time period and the place; however, there is no feeling of presence in time. One AI-generated image gives you one frame with no story in it. Thus, Yatra falls somewhere in between, giving you an opportunity to walk through the historical place itself, see it, read information about the life there and move from scene to scene.

What it does

  1. Search -user selects a place and era on the frontend.
  2. Global cache check - MongoDB is checked for an existing system-owned walkthrough for that exact place/era. If found, served instantly.
  3. RAG retrieval - on a cache miss, a local Chroma vector store is queried for historically grounded context.
  4. Agent generation - a Gemini agent turns that context into a structured 3–5 stop JSON walkthrough (second-person narration, daily-life facts, continuity-aware image prompts).
  5. Image generation - each stop's image prompt runs through a fallback chain: Imagen 4 (fast → standard → ultra) → Pollinations.ai → transparent placeholder.
  6. Persistence - the finished walkthrough (including Base64 image URLs) is saved to MongoDB under user_uid="system" and returned to the frontend.

Users can also save a copy to their own account (clone, not move) and generate an 8-character share slug for public, unauthenticated viewing.

How we built it

  • Backend: FastAPI, with a RAG layer (OpenAI embeddings over a curated 25-place/era corpus, indexed in Chroma) feeding historical context into an agent step that generates the structured walkthrough JSON.
  • Agent: runs on Google Gemini, with a same-provider fallback across Gemini's own model tiers for resilience against quota limits.
  • Image generation: a multi-tier fallback chain, landing on a working free-tier image source when primary options weren't accessible namely Pollination
  • Persistence: MongoDB Atlas completed walkthroughs are saved verbatim, so reopening or sharing one replays the exact original result rather than regenerating it.
  • Auth: Firebase (Google + email/password).
  • Frontend: React/Vite, merged from two independently-built pieces , a landing/auth experience and the core walkthrough dashboard into one integrated web- app during the build.

Challenges we ran into

  • Provider quota limits forced real architecture decisions under time pressure we ended up routing the agent through Gemini and building a same-provider fallback chain rather than a single hardcoded model call, which made the pipeline meaningfully more resilient than our original design.
  • Image generation reliability was a recurring fight primary providers weren't consistently accessible, so we built a multi-tier fallback with a placeholder as a last resort, ensuring a single failed image never breaks the whole walkthrough.
  • Merging two independently-built frontend pieces (built by different people, different tooling versions, different design systems) into one coherent web-app without regressing either required an isolated dependency upgrade pass before the actual merge to keep the two changes separable.

Accomplishments that we're proud of

  • Agent JSON validity held at or above our >90% target across a real batch of historical place/era pairs, not just a handful of manual tests.
  • Firebase Auth and MongoDB persistence both work correctly, including correct per-user isolation on saved walkthrough history.
  • Kept the core loop demoable throughout, even while absorbing a full extra role's worth of scope with a smaller team than planned.

What we learned

Coordinating a multi-stage AI pipeline madee up of RAG retrieval, structured generation, and image synthesis all depending on each other's output, turned out to be harder than treating each stage in isolation. Small inconsistencies early in the chain (a vague retrieved context, an ambiguous image prompt) compound by the final stop. We learned to invest heavily in structuring the intermediate JSON contract between stages so failures were caught early rather than surfacing as a garbled walkthrough downstream.

What's next for Yatra

  • Wire up the supported places list - connect the frontend to GET /api/walkthrough/places so users can't submit unsupported place/era combinations and hit a 400.
  • Finish sharing — stabilize public link generation and add a proper "Walkthrough Not Found" state for broken/invalid share links.
  • Move off Base64 image storage - upload generated images to Firebase Storage and store only the CDN URL, instead of bloating MongoDB with data URIs.
  • Migrate off deprecated Imagen models - move to imagen-3.0-generate-001 or native Gemini image generation ahead of the late-2026 deprecation.
  • Expand the historical corpus - support more place/era combinations beyond the initial set.
  • Mobile-responsive layout.
  • Dedicated viewer page - a proper standalone view for walkthroughs instead of the current modal-based flow.
  • User profiles - persistent identity beyond auth, surfacing saved walkthroughs and history.
  • Conversational agent - a chat interface for asking follow-up questions about a stop's historical context.
  • Gamification - mark visited places, track progress across walkthroughs.

Built With

Share this project:

Updates