Inspiration
Everyday tasks can become frustrating or overwhelming for people with memory and cognitive impairments. Remembering what comes next, recognizing someone, or figuring out an unfamiliar device can turn a simple moment into one that requires help from a family member or caregiver.
We built SImpllife around a simple idea:
AI should not replace what people can do. It should help them keep doing it themselves.
As AI agents become increasingly focused on completing tasks for people, we wanted to explore the opposite direction: using AI and augmented reality to understand when someone needs support and provide just enough guidance for them to continue independently.
What it does
SImpllife is an AI-powered augmented reality cognitive assistant designed to help people navigate everyday life more independently.
Using a Meta Quest headset, SImpllife understands what the wearer is looking at, what they are trying to accomplish, and what is happening in their environment.
A user can ask:
"Hey Muse, how do I do this?"
while looking at an object.
SImpllife captures the wearer's view, uses multimodal AI to understand the scene, retrieves trusted instructions, and guides the user through the task directly in augmented reality.
Instead of simply displaying a list of instructions, SImpllife creates a continuous feedback loop:
Observe → Understand → Guide → Observe → Adapt
As the user performs each action, AI can recognize changes in the physical environment and determine when they are ready for the next step.
Spatial AR can highlight the exact object or control the wearer needs to interact with. Muse acts as an embodied guide that can speak, gesture, demonstrate an action, or present a virtual screen containing a step-by-step tutorial or family-recorded video.
Families and caregivers can use our companion application to create and approve people, memories, routines, instructions, photos, videos, and voice messages.
Our guiding principle is:
AI understands the moment. Humans provide or approve the truth.
How we built it
SImpllife combines Meta Quest 3, Unity, multimodal AI, spatial AR, and a caregiver web platform.
When the wearer asks Muse for assistance, the Quest captures the user's current view and sends the visual context and request to our backend.
Gemini analyzes the scene to understand relevant objects, activities, and physical states. SImpllife then connects that understanding with trusted information stored through the caregiver platform.
We designed the system so that AI can determine things such as:
- What the user is looking at
- What activity they may be performing
- What state an object is currently in
- What step of a task they are on
- Whether an action appears to have been completed
However, personal information such as relationships, memories, routines, and caregiver instructions comes from trusted human-provided or human-approved data.
On the headset, SImpllife brings that guidance back into the physical world through spatial AR. Muse can explain the next step, gesture toward the relevant object, display a tutorial, or provide additional assistance when the wearer is stuck.
The caregiver application allows families to manage people, relationships, memories, everyday routines, step-by-step instructions, photos, videos, and voice recordings.
Together, these systems create a closed loop:
See the world → Understand the situation → Retrieve trusted guidance → Help the user act → Observe the result → Adapt
Challenges we faced
One of our biggest challenges was deciding what AI should and should not be allowed to do.
Generative AI is powerful, but confidently generating the wrong personal memory, relationship, or instruction could make an assistive system less trustworthy. We therefore separated AI-generated understanding from human-approved personal knowledge.
Another major challenge was connecting AI understanding to the physical world.
Recognizing an object is only the beginning. SImpllife needs to understand the state of that object, determine which step the user is on, provide useful spatial guidance, and then recognize when the physical world has changed.
We also had to work through the realities of real-time AR, including Quest camera capture, networking between the headset and our backend, latency, spatial alignment, speech interaction, and designing assistance that is useful without becoming overwhelming.
What we learned
We learned that one of the most interesting applications of AI for accessibility is not necessarily having an agent perform more actions automatically.
Sometimes the better question is:
How can AI help someone remain capable of performing the action themselves?
We also learned how powerful context can be.
A traditional reminder can tell someone what to do. Multimodal AI can potentially understand what someone is looking at, what they are currently doing, what has already happened, and what assistance would be most useful next.
Finally, building SImpllife taught us that personalization and trust cannot simply be added to an assistive AI system afterward. They need to be part of its architecture from the beginning.
What's next
We want SImpllife to become an adaptive cognitive layer over the physical world.
Future versions could support more environments and routines, improve spatial understanding, personalize how much assistance each person receives, and allow Muse to intelligently move between subtle reminders, spatial highlights, demonstrations, and family-created tutorials.
Our goal is not to build an AI that lives someone's life for them.
Our goal is to give people the confidence and support to keep living it themselves.
Log in or sign up for Devpost to join the conversation.