Inspiration
Matt and I kept coming back to one idea: our current prosperity did not appear overnight. It is the sum of countless moments, places, and people across history.
Most people can read about history, but very few can feel what a place and time might have been like. We wanted to build a way to step into those moments and make history feel alive, personal, and accessible.
What we built
We built an AI-guided exploration experience:
- Pick any country, province, city or location on a 3D globe
- Talk to an AI guide (through voice) to choose a year and area of the world you want to explore
- Warp to hyperspace while the world is generating, with listening to music tailored to the selected time, place, and culture
- Arrive inside a rendered 3D world generated for your exact coordinates and chosen moment in history
- Explore the world while your guide interprets what you’re seeing and shares the historical context behind it.
The full experience stays inside our app. No redirect is required to explore.
How we built it
- Frontend: Vite + React + TypeScript + Three.js + Spark.js
- Globe phase: interactive Earth selection for location + year (with infinite zoom-in)
- Voice loop (Gradium STT/TTS): low-latency, interruptible conversation so users can cut in at any time and still have a natural back-and-forth
- Core logic (Gemini API): Used as the decision layer for guide behavior, prompts, and responses across every phase
- Gemini tool calling: drives globe suggestions, era-aware music, and world creation triggers so the map, audio, and generation pipeline stay in sync
- World generation: World Labs API with prompts that anchor the location, year, cultural context and users interests deteremined from the conversation.
- World rendering: Spark.js + Three.js Gaussian splat rendering pipeline
- UX pipeline: starfield + globe -> hyperspace -> loading narration -> exploration
Challenges we faced
- Managing conversation state during interrupts: if VAD split one user thought into multiple turns, Gemini could fire twice and create overlapping responses
- Making WebSocket streaming robust on the frontend: without strict queueing and response-state control, audio/text events could previously arrive out of order and feel jumpy or buggy
- World renderer: splat artifact rendering issues in Spark.js integration (still working on fixing this)
- Matching color and visual quality to the reference World Labs viewer
What we learned
- Adding strong state transitions (landing page -> globe -> worldview) made the journey more immersive
- We found there was a tradeoff between very low STT latency and system stability
- Learned to use tool calling much more effectively for structured actions like location selection and dynamic messaging.
- Orchestrating voice, world generation, and rendering required robust state management with graceful recovery paths.
- The best loading experience is still a meaningful experience: context-aware music and messages keep users inspired.
- Experiencing it ourselves, showed us just how special this is and how it creates a new way to learn history and culture.
Built With
- cartodb
- fastapi
- gemini
- globegl
- gradium
- python
- react
- spark.js
- three.js
- typescript
- vite
- websockets
- worldlabs


Log in or sign up for Devpost to join the conversation.