Atomic Tales

Project Cover
Characters creation BTS
Canister and Hyperfuel creation BTS
Call with Jesse to understand about neurodivergent people BTS
Pip and spaceship creation BTS
Island creation by hand sculpting and painting BTS
Level islands to indicate & summarise learnings
Atomic structure to show arrangement of electrons, protons and neutrons for each element

Inspiration: Empowering Neurodivergent Learners

Atomic Tales was born from a desire to bridge the gap in teaching chemistry to neurodivergent individuals, specifically those with ADHD, Down Syndrome and Autism.

Traditional chemistry education is often abstract, hard to visualise, text-heavy, and disconnected from the practical realities of daily life. We saw an opportunity to use Spatial AR and Spatial AI to transform Organic Chemistry from a daunting set of rules into an embodied, tactile, and story-driven journey grounded in a familiar home environment.

What it does: A Recipe for Universal Understanding

Atomic Tales is a Spectacles-first AR experience that reframes organic chemistry as a series of missions. Once users don the Snap Spectacles they are greeted by Pip, a cute pizza-shaped space alien who has crash-landed in the user's space.

Narrative-Driven Campaign: In "Story Mode," Pip represents a neurodivergent character who runs out of fuel and lands in the user's world which is confusing and difficult to navigate. The player helps Pip feel safe, understand our world, and rebuild his spaceship's hyperfuel core through five progressive levels represented by floating islands.
Spatial AI Discovery Scanner: Instead of looking at 2D organic chemistry diagrams, players search their environment for elements from their everyday objects. Powered by Gemini Live video streaming and structured image polling via the Snap Remote Service Gateway, the scanner identifies target objects (e.g., finding water in a cup, or carbon in a wooden chair) and projects corresponding interactive atom prefabs directly above them.
Bite-Sized Practical Science: Learning is broken down into 5 progressive levels:
- Level 1 (Water): Scan for H₂O, split the molecule, and collect H + H + O atoms.
- Level 2 (Carbon): Scan for Carbon (C), zoom into its atomic structure—nucleus, protons, neutrons, and electrons—and witness a Carbon Monoxide (CO) bonding demo.
- Level 3 (Remaining Elements): Scan and collect Hydrogen (H) and Nitrogen (N) to learn about atomic structure and reactions.
- Level 4 (Fuel Lab): Release all collected atoms from the bracelet and assemble them into Nitromethane (CH₃NO₂) to synthesize the glowing hyperfuel core.
- Level 5 (Spaceship Sendoff): Drag the hyperfuel core into Pip’s ship, restart the thrusters, and wave goodbye.

Sandbox / Pizza Bar Mode: An epilogue workshop where players travel through the portal to Pip’s home planet. Here, chemistry becomes endless recipe play at a virtual Pizza Bar, allowing players to freely collect elements, mix toppings, and experiment with reactions.
Tactile Engagement & Bracelet Collector: Using the Spectacles Interaction Kit (SIK), users physically pinch floating atoms to pull them out of the environment and store them in slots on an interactive wrist bracelet.

How we built it: Inclusive Spatial Design

We prioritized accessible, multi-sensory feedback to keep the experience low-stress and rewarding.

Platform & Tech: Developed for Snap Spectacles using Lens Studio and TypeScript. We integrated Gemini models (gemini-2.5-flash-lite and gemini-2.0-flash-live-preview-04-09) with the Snap Remote Service Gateway.
Dynamic Spatial Layout: To manage the user's field of view, the experience uses a FloatingIslandController that dynamically instantiates and hovers three reusable island prefabs (Teal, Purple, and Pink) to match the current story level.
Audio & Dialogue System: Pip's speech uses fully local, pre-cached MP3 dialogue assets to eliminate network latency. An event-driven SoundController coordinates ambient tracks, UI sound effects (like buttons and portals), and Pip’s voice lines.
State Persistence: An AtomicTalesSaveStore writes progress to storage after every level, allowing users to resume exactly where they left off if the lens restarts.

Challenges we overcame: Scaling Complexity

Asynchronous AI Latency: Cloud AI responses can take time, during which a user might move their head. We solved this by implementing a DepthCache that captures synchronized color and depth camera frames. When Gemini returns a bounding box, we project the coordinates using the cached depth frame rather than the current frame, ensuring precise world-space placement of atom prefabs.
Motor Control Accessibility: To accommodate fine motor challenges, we tuned the interaction thresholds within SIK, scaled up interactive atoms for easier targeting, and implemented a smooth proximity-snapping system for molecule assembly in the Level 4 fuel lab.

Accomplishments that we're proud of

We are most proud of our Depth-Projected Spatial AI Scanner. By merging real-time depth mapping with Gemini's vision-reasoning capabilities, we created a magic-lens effect where physical objects in the room genuinely seem to "release" their core chemical elements into the room.

What we learned: The Power of the "Why"

Designing for neurodivergence taught us that grounding abstract topics in physical actions (like pinching, pulling, and slotting) and familiar motifs (like cooking recipes and helpful alien companions) replaces the anxiety of rote memorization with intuitive spatial memory.

What's Next for Atomic Tales

Replacing Prototype Placeholders: Our next step is to transition the visual placeholders for the Level 4 assembly vortex and the Level 5 ship launch into final interactive behaviours.
Expanding Pizza Bar Recipes: We aim to build out additional recipe datasets for the Sandbox Mode so players can experiment with a wider range of organic compounds.
Applying the Framework: Leveraging this modular state-machine and scanner pipeline to create new spatial learning applications for other scientific disciplines.