-
-
Welcome to Mnemosyne — an immersive magical library for spatial learning and memory.
-
Interact with virtual books and objects using natural hand tracking on Meta Quest 3.
-
Read and explore knowledge through immersive, interactive book pages.
-
Explore interactive knowledge books containing facts, explanations, and memory prompts.
-
Discover and select knowledge collections from a magical bookshelf.
-
Museum of Strange Objects — one of three built-in knowledge collections in Mnemosyne.
-
Associate knowledge from interactive books with spatial memory locations in mixed reality.
-
Color-coded spatial markers help users distinguish memory locations and their binding states.
Inspiration
What if knowledge could become part of the world around us?
Mnemosyne was by the ancient method of , also known as the memory palace technique. This method helps people remember information by associating it with familiar locations.
I have always been by how people learn, remember, and organize knowledge. When I began mixed reality, I wondered: What if we could turn the memory palace from an technique into something we could actually see and with?
Instead of isolated facts on a screen, imagine walking into a magical library, picking up a book, and placing memories throughout a environment.
I wanted to the cognitive principles of spatial memory with the immersive possibilities of XR, making learning feel more like than .
That idea became Mnemosyne, named after the Greek goddess of memory.
What it does
Mnemosyne is a mixed-reality learning prototype developed for Meta Quest 3. It transforms knowledge into an interactive experience built around books, locations, and memory associations.
The core experience follows four stages:
- Explore a magical library
enter an immersive environment architecture and interactive books. They can pick up books, different knowledge collections, and what they want to learn.
The current prototype contains three built-in knowledge collections with 24 knowledge items in total.
- Build associations
associate knowledge items with spatial memory . Through hand-based interactions, they can locations, confirm , decisions, and continue unfinished sessions.
The competition prototype currently uses MRUK JSON-based room to spatial locations. with actual scanned room data remains under development.
- Practice and memory
After creating associations, users can practice identifying the locations connected to their knowledge.
The practice system introduces candidates , increasing the difficulty across rounds. Only previously memory locations are included.
- Learn with a guide
Mnemosyne also features a voiced guide who provides , responses, and knowledge explanations.
The guide uses prerecorded dialogue with triggers and , rather than real-time AI generation.
features include UI and , visual effects, sound , local progress , and JSON backup support.
The goal is to make remembering information feel like interacting with a world rather than studying a static document.
How we built it
Mnemosyne was developed independently over approximately two weeks, beginning on November 4, 2026.
As a first-year computer science student with limited practical XR development experience, I explored several technical approaches before establishing the final architecture.
Phase 1 — WebXR and immersive interaction
I began with Three.js and WebXR, creating 3D scenes and interactive knowledge objects.
I then explored Meta's Immersive Web SDK (IWSDK), experimenting with hand tracking, grabbing, and immersive interaction systems.
Phase 2 — Unity and Meta XR SDK
Next, I experimented with Unity, learning about XR rigs, hand visualization, Android builds, and to Quest .
These experiments helped me understand the challenges of building applications for standalone XR devices.
Phase 3 — Native Android development
The current Mnemosyne application was built using Kotlin and Meta Spatial SDK.
Its main technologies include:
Kotlin and Android for application logic and lifecycle management.
Meta Spatial SDK 0.14.0 for immersive scenes and spatial .
Meta Interaction SDK (ISDK) for hand-based interaction.
Meta MRUK for spatial geometry and memory location generation.
Jetpack for .
Meta Spatial Editor and Blender for 3D scenes, models, and .
JSON, SharedPreferences, and Android storage for knowledge and learning progress.
Python and text-to-speech tools for development and prerecorded voice .
The application combines authored scene assets with dynamically created interactive books, spatial , and UI elements.
I used AI-assisted development tools to help explore APIs, and code, and investigate bugs. I directed the product design, selected technical approaches, the systems, and tested the results.
This project required learning not only how to write XR code, but also how to build, , package, and a real application to Meta Quest 3.
Challenges we ran into
- Learning multiple XR development
My first challenge was choosing a suitable development approach while still learning the of XR.
I explored WebXR, IWSDK, Unity, and native Android development. Each introduced different tools, interaction models, and debugging challenges.
I learned that choosing a framework is not simply about its , but also about the practical of the application and the development process.
- Scene loading, , and unexpected crashes
Several bugs came from interactions between scene and runtime code.
One particularly difficult crash occurred because an expected scene was missing from the scene data.
Solving it required investigating exported assets, scene initialization, object creation timing, and animation .
Other problems SDK paths, missing , and interaction hitboxes.
These experiences taught me to investigate logs systematically and isolate the actual causes of failures.
- Accessing spatial room information
One of the most challenging problems Meta Quest's scene APIs.
Although the required permission was granted, attempts to room information failed with an application trust error:
ERROR_PACKAGE_UNTRUSTED (-10035)
I investigated permission handling, , and application trust, including with an official sample.
Because the issue remained unresolved, I decided to use MRUK's room for the while preserving a separate path for real-room access.
This was an important lesson in managing : sometimes the best engineering decision is to isolate a blocking problem and continue developing the parts that can be independently.
- Designing interactions for people with VR
Near the end of development, I invited my roommates to try Mnemosyne.
Their reactions surprised me.
Some did not understand how to books, how to with buttons while holding an object, or what they were supposed to do next.
Features that felt obvious to me as the were confusing to first-time users.
Based on their , I the initial , improved , added , and revised the book- process.
The testing also revealed that the experience needed clearer and more engaging interactions.
These are , and the latest changes still require further user .
that we're proud of
My greatest accomplishment is turning an idea into an application that can run on an actual XR .
Before this project, I had never independently developed an XR application of this .
Over approximately two weeks, I explored multiple development frameworks and built a containing:
An magical library with 3D elements.
Three knowledge collections containing 24 knowledge items.
Hand-based book and spatial memory binding.
practice and memory-location testing.
A voiced guide, UI, and .
Local learning progress and backup .
I also gained experience in and testing.
The JVM test results contain 27 passing tests with no reported failures or errors, covering aspects of memory binding, practice , state, and other application logic.
Earlier versions were tested on Meta Quest 3, including scene and parts of the hand interaction system. The latest build still requires complete on-device testing.
Finally, I successfully built and Mnemosyne 1.0.1 to the Meta Competition release channel.
For me, this represents more than completing a . It is my first experience taking an immersive application from an idea through development, , user testing, and release .
What we learned
This project completely changed how I think about development.
First, I learned that a feature is not the same as designing a good experience.
I spent time making interactions work, but user testing showed me that technical alone does not make an application intuitive.
A feature is only useful when people can understand how and why to use it.
Second, I learned the importance of managing .
My original vision was much larger than what I could build within the competition period.
I had to the essential experience, simplify ideas, and find alternatives when technical limitations blocked progress.
Third, I learned how valuable is.
From Gradle problems to scene-loading crashes and spatial API errors, I learned to use logs, isolate , test assumptions, and confirmed causes from .
Finally, I learned that building something real requires much more than knowing how to code.
It requires product design, technical research, asset integration, testing, communication, and the willingness to your own assumptions.
As a first-year computer science student, Mnemosyne has been one of the most and learning experiences I have .
It is not a finished product, but it is a beginning.
What's next for Mnemosyne
I plan to continue developing Mnemosyne beyond this competition.
My immediate priorities are to improve , complete on-device testing, and make the experience easier for new users to understand.
My longer-term plans include:
Real-world spatial integration: scene access issues and investigate spatial anchors so memories can remain associated with real environments.
Better : Replace text-heavy with demonstrations and clearer visual .
Deeper memory exercises: Move beyond location toward exercises that actual knowledge recall.
More expressive memory objects: Represent knowledge through 3D objects and spatial storytelling.
AI-assisted learning: Explore knowledge organization, , and more content creation.
User-centered : Conduct structured testing to usability, engagement, and learning outcomes.
, I hope Mnemosyne can grow into a personal spatial knowledge environment where people can build, explore, and their own memory palaces.
I started this project with a question: What if memories could have a place in the world around us?
Mnemosyne is my first step toward answering it.