GeminiAtlas Snap Spectacles

Scan the world. AI explains. AR displays. Learn by moving.

Comment

Inspiration

What if Pokémon GO was also a social experience that fuels your curiosity?

A huge part of short-form social media today is being glued to your phone, passively scrolling, with our attention constantly fragmented – one of the main causes of the rise in learning difficulties worldwide, especially for youth in underserved regions. GeminiAtlas opposes every aspect of the "death scroll". It encourages you to go out and discover the world, to be a curious explorer, to connect with people across time and space, and to live and grow through what you learn.
Our mission: Turn curiosity into a daily habit and make learning as addictive as scrolling.

What it does

GeminiAtlas is a Snap Spectacles lens that turns your everyday environment into a personalized, social, AI-powered learning experience. It’s built on several pillars designed for exploration, education, and connection: 1. Align the sky with you: the Constellations Lens. This is an augmented reality Snap Lens that overlays constellations directly onto the real stars you’re seeing. Built for anyone, anywhere.
With Lens Studio, I integrated 3 key controls:

  1. Location: The user changes country or city. The sky recalculates in real time for accurate astronomy.
  2. Compass: The Lens orients with you. You look up North, it displays the Big Dipper instantly.
  3. Voice control: The user says "Orion" or "Cassiopeia". The Lens calls up the constellation and highlights it with facts. The result: The sky is no longer abstract. It becomes an interactive astronomy textbook accessible to students without telescopes. 2. Read for you: the Bookwoom Lens. Bookwoom is a Lens for Snap Spectacles that democratizes access to knowledge.
    The concept is simple: the user frames a book with their hands.
    With Lens Studio + Gemini Vision, GeminiAtlas analyzes the cover live. In 2 seconds, I surface: author, summary, key themes, and similar books. The text leaves the cover and becomes a reading card you view at a glance.
    The result: A library becomes a living database. Perfect for students and lifelong learners. 3. Show you the Globe: Your Memory Palace. This is the heart of GeminiAtlas. Every card you capture with Constellations or Bookwoom has a position, a subject, a memory. I organize them spatially around you in AR.
    You turn your head: Noguchi in Seattle. Another turn: innovation in Cotonou.
    Your long-term memory becomes a landscape. The more you save, the more the experience becomes your best learning partner. This spatial learning boosts retention by 3x. 4. Capture and remix! Capture any element in your environment with a simple two-hand pinch, and GeminiAtlas turns it into a short, surprising Curiosity card based on a random interest of your choice. You can then start a conversation with the Gemini AI assistant about each card to get details and learn more.
    We have repeatedly found that the most memorable things we learn are not the ones we study in a structured way, but the ones we discover by chance.
  4. The intersection of two disciplines often leads to unexpected discoveries.
  5. Our interests are the best guides for expanding our understanding.
  6. Exploration makes you more curious and a better learner; it’s a virtuous circle. 5. Explore and share! The cards are geolocated and designed to be shared. Join your hands in front of your chest – a symbolic gesture that "discovers the world" – and the lens analyzes your location to reveal a universe of cards left by you and others. Save, comment, follow, or share your own capture!
  • When you find an interesting object, discover what others have learned about it – perhaps through an interest you had never considered before.
  • When you visit a new place, explore it with those who came before you, thus sharing a moment across space and time. Building a global community of curious minds. 6. Relive and remember A living, evolving memory palace in the palm of your hand, where all your saved cards are stored. The AI organizes everything spatially, by subject, relevance, and by what it learns about you. The more you save, the better the experience understands how you learn and suggests new paths. 7. Collect and battle! Take on the challenge and learn together by competing with friends in a Q&A game. Questions are randomly generated from both players' cards using GPT-4o; the one with the most cards and who knows them best wins. An AI host, with offbeat humor and audio-only, reads each question aloud and reacts to your answers to keep learning fun.

How we built it

GeminiAtlas is a Spectacles lens built in 9 days with Lens Studio for the Rampable Hackathon.
All our AI models are processed securely by Snap’s Remote Services Gateway (RSG) for low-latency performance. Tech Stack:

  1. AI Brain: Gemini Live for voice conversation, OpenAI Vision for image understanding, GPT-4o for battle question generation
  2. AR Engine: Lens Studio, SpectaclesSyncKit for multiplayer
  3. Architecture: Global global.* singletons for state management. Streaming speech as 24 kHz PCM. Guided state machine for seamless globe-to-card transitions. The capture pipeline performs a two-hand crop to generate an OpenAI Vision caption. Battle mode is managed by the host on SpectaclesSyncKit, with questions generated from captured cards and broadcast between the two players in real-time.

Challenges we ran into

Building for AR + AI + Social was not easy. Most difficulties involved coordination – between sessions, devices, and clocks.

  1. RSG Session Limit: RSG only keeps the most recent session active – two simultaneous Gemini sessions drop the connection. Solution: We built a voice time-slot sharing system so assistant and host take turns.
  2. Microphone Stability: The shared microphone is destructive and can lock up. A failed start leaves the provider "started but out of service". Solution: Implemented a cyclical stop-restart mechanism and monitoring system for recovery.
  3. Multiplayer Sync: Because device clocks aren’t comparable, the leaderboard had to be based on the server clock. Also, UIkit buttons only fire on startup, so we rebuilt the interaction logic to avoid dead buttons.

Accomplishments that we're proud of

We’re proud that the assistant looks like a character, not just a chatbot: a strong personality, an expressive sphere that reacts to sound, real-time text captions, all running via the gateway with no on-device work. The globe-to-card navigation is exactly as we imagined: a single continuous gesture, from orbit to a street you can walk, with the transition invisible by design. Card discovery is truly magical: a plane wave in the room and the cards appear as if by magic at every touch. And we shipped a real two-player quiz, latency-tolerant, with an adaptive AI host that knows not to mock a player who’s already struggling. That’s empathy in AI.

What we learned

The through-line of everything we learned: decouple thought from voice and make the deterministic parts boring.

  1. Modularity Wins: Our line banks, tool declarations, and query logic are fully model-independent and easily testable; agents only own the live session.
  2. Smart Caching: We freeze what never changes. The 37 fixed Cosmos cards send pre-written questions instantly and for free, while only user-captured cards require an API call. This saves cost and latency.
  3. Deterministic AR: We keep placement calculations pure. Every card scatter is seeded by the card ID, so markers never drift between frames, zooms, or sessions.

What's next for GeminiAtlas

More people, more curiosity, more impact. Currently, the globe offers Tokyo, Seattle, and Los Angeles with a universe of 37 cards and 13 initial interests. Roadmap:

  1. Scale: Expand the map to 50+ cities including Cotonou, Dakar, Nairobi. Add 100+ themes.
  2. Social: Deeper profiles, ability to follow creators, themed learning paths through a neighborhood.
  3. Education: Partner with schools and libraries to use GeminiAtlas for field trips and interactive learning.
  4. Battle 2.0: Grow Battle mode beyond two players, with small-group matches and themed decks. Long term vision: We want GeminiAtlas to become a living, shared atlas of human curiosity: every wall, every dish, and every street corner annotated by someone who found it interesting, waiting for the next person to pass by. Making the world the classroom.

In conclusion:
GeminiAtlas + Snap Spectacles + Lens Studio = learning by moving, sharing, playing.
The buzzer is done waiting. Ready to start exploring?

Built With

Share this project:

Updates

Submission history