Memoryze

Inspiration

The growing impact of memory-related conditions on millions of lives worldwide inspired us to create Memoryze. We recognized that while traditional sticky notes help people remember important information, they can be easily lost or overlooked. By leveraging mixed reality technology, we saw an opportunity to create a more interactive and intelligent memory aid system that could seamlessly integrate into people's homes while providing the familiarity of sticky notes.

What it does

Memoryze transforms living spaces into interactive memory support environments using the Meta Quest 3's mixed reality capabilities. Users can place virtual sticky notes throughout their home environment, creating context-aware reminders. Using OpenAI's Whisper API for speech-to-text conversion, users can verbally interact with their notes and a built-in chatbot that helps them locate and manage their virtual reminders. This creates a hands-free, natural interface that's particularly beneficial for elderly users or those with cognitive challenges.

How we built it

We developed the application using Unity for the Meta Quest 3 platform, implementing mixed reality features to blend virtual elements with the physical environment. The system integrates OpenAI's Whisper API for speech recognition, allowing for natural voice interactions. We created a spatial mapping system to accurately place and persist virtual sticky notes in the user's environment, and implemented a chatbot system that can understand and respond to queries about placed notes.

Challenges we ran into

Optimizing the spatial anchoring system to ensure sticky notes remained precisely where users placed them between sessions
Implementing reliable voice recognition in potentially noisy home environments
Creating an interface that would be intuitive for users who might not be technically savvy
Balancing the visual presence of virtual notes to ensure they're noticeable without being intrusive

Accomplishments that we're proud of

Successfully creating a seamless mixed reality experience that feels natural and helpful
Developing an accessible voice interface that works reliably for different users
Implementing a spatial memory system that effectively maintains note positions
Creating an application that genuinely helps people with memory challenges maintain independence

What we learned

The importance of user-centric design when creating applications for users with cognitive challenges
Technical insights into mixed reality development and spatial computing
How to effectively integrate voice recognition and natural language processing in XR environments
The complexities of creating persistent mixed reality experiences