Yaad

Yaad — No One Deserves to Forget. Everyone Deserves to Remember.

Inspiration

We’ve all experienced the frustration of forgetting where we left our keys, what we were supposed to do, or where we last encountered someone. But for people living with dementia and individuals experiencing memory difficulties, these everyday moments can become overwhelming challenges that gradually affect their independence, confidence, and quality of life.

We envisioned a world where technology could help us remember without requiring us to constantly interact with a screen.

What if your Meta glasses could remember where you left your belongings? What if they could recognize your everyday essentials, remind you before you walk away without them, and help you find them again?

That’s the vision behind Yaad — because we believe memory assistance should empower people, not make them more dependent on others.

What it does

Yaad is an AI-powered memory assistant envisioned primarily for Meta smart glasses that transforms everyday visual experiences into persistent, searchable memories.

Its core capabilities include:

• AI-Powered Belongings Recognition: Uses Meta’s Muse API to explore visual understanding and identify everyday essentials such as keys, wallets, and phones within a user’s surroundings.
• Visual Memory Mapping: Associates belongings with the visual context and surroundings in which they were last observed.
• Voice-Based Memory Retrieval: Allows users to ask natural questions like “Where did I leave my keys?” and retrieve their last recorded observation, including contextual information and images.
• Departure Reminders: Demonstrates how users could be reminded to check important belongings before walking away.
• Virtual Meta Glasses Experience: An integrated website simulator demonstrates the envisioned glasses interaction without requiring physical hardware.
• Spatial Computing Prototype: A Unity-based prototype explores how memory assistance could translate into an immersive, glasses-native experience.

Our broader vision includes contextual proximity reminders and navigation to belongings, making everyday memory assistance even more seamless.

How we built it

Yaad combines AI-powered visual understanding, full-stack web engineering, secure cloud infrastructure, and spatial computing.

AI & Meta Integration: We integrated Meta’s Muse API to explore AI-powered visual understanding, extracting meaningful information from visual inputs and using it as the foundation for our contextual memory pipeline.

Web Application: Built using Next.js 14, React, TypeScript, and Tailwind CSS, the application provides an interactive interface for capturing, organizing, and retrieving memories.

Backend & Persistence: Supabase powers authentication, PostgreSQL-based persistent storage, Row-Level Security (RLS), and secure image storage through private buckets and signed URLs.

Spatial Computing: We developed an exploratory prototype using Unity 6, Meta XR SDK, and OpenXR to investigate how Yaad’s memory interactions could translate into a spatial, hands-free environment.

Memory Pipeline: Visual observations are connected to contextual metadata and persistent records, enabling users to retrieve previously recorded belongings through natural-language queries.

We developed complementary prototypes: a functional web application demonstrating the core memory experience and a Unity spatial prototype exploring the future of glasses-first interaction.

Challenges we ran into

One of our biggest challenges was translating an abstract concept like human memory into a technically meaningful and intuitive experience.

• Contextual Understanding: Identifying an object is not enough. Yaad needs to associate it with meaningful surroundings so users can actually find it again.
• Persistence & Privacy: Building reliable memory storage required secure authentication, user-isolated database access, and protected image storage.
• Hardware Limitations: We did not have physical Meta glasses available for development, so we explored the experience through a virtual glasses simulator and Unity-based spatial prototyping.
• Bridging Web and Spatial Computing: Translating an established web-based interaction model into a spatial environment required exploring different interaction patterns and technical approaches.
• Scope & Feasibility: We had to distinguish between the core functionality demonstrated in our prototypes and the broader vision of real-time contextual reminders and navigation.

Accomplishments that we’re proud of

• Building a functional AI-powered memory assistant with persistent data storage and contextual memory retrieval.
• Integrating Meta’s Muse API into our visual understanding and memory exploration pipeline.
• Implementing authentication, database persistence, user isolation, and secure image storage using Supabase.
• Developing a virtual Meta glasses simulator to demonstrate the envisioned user experience without requiring physical hardware.
• Building a Unity 6 spatial prototype using Meta XR SDK and OpenXR to explore glasses-native interactions.
• Bringing together AI, full-stack engineering, and spatial computing into a single cohesive product vision.
• Designing Yaad around a meaningful real-world problem: helping people preserve independence and remember what matters.

What we learned

We learned that building AI-powered products is not simply about integrating a model. The real challenge lies in turning AI capabilities into experiences that are intuitive, contextual, reliable, and genuinely useful.

We explored how visual AI can become the foundation of a persistent memory system, how spatial computing can change the way people interact with technology, and how privacy and secure data handling are essential when building systems around personal memories.

Most importantly, we learned that impactful technology does not always need to replace human abilities. Sometimes, it simply needs to support them at the right moment.

What’s next for Yaad

Our vision for Yaad extends far beyond the current prototypes.

• Real-Time Glasses Integration: Move toward a fully functional experience on Meta smart glasses, enabling natural, hands-free memory capture.
• Context-Aware Reminders: Explore proximity-based reminders that notify users when they are leaving important belongings behind.
• Visual Navigation: Develop contextual guidance to help users return to the last known location of misplaced items.
• Enhanced Spatial Experience: Refine our Unity prototype into a more polished, intuitive spatial computing experience.
• People Memory: Explore consent-based memory assistance for remembering names, relationships, and meaningful personal context.
• Accessibility & Dementia Support: Work toward making Yaad a supportive everyday companion for people living with dementia and others experiencing memory difficulties, helping preserve autonomy and reduce reliance on caregivers.

Ultimately, we want Yaad to become more than a tool for finding lost belongings. We want it to become a bridge between technology and human memory — helping people navigate life with greater confidence, independence, and peace of mind.

Yaad. No one deserves to forget. Everyone deserves to remember.

Built With

• built-with-meta-muse-api
• meta-xr-sdk
• next.js
• openxr
• supabase
• typescript
• unity-6