Memory Palace

Immersive memory recall for dementia care

Comment

Inspiration

There's a recurring image in films about dementia — a loved one patiently retelling cherished memories, desperate to keep someone tethered to who they are. For many, that's not a movie scene. It's Sunday afternoon. It's a phone call that ends in silence.

My grandfather recently began to develop memory loss, and with them, pieces of the moments we built together. Watching that happen is its own kind of grief — and it's what pushed us to act.

We built a tool for people like him. Traditional memory exercises can feel cold and clinical, so we wanted something different: reminiscence that feels immersive, emotionally resonant, and deeply personal. A place where meaningful life moments aren't just recalled — they're revisited. Patients can explore the spaces, stories, and feelings that shaped who they are, and reconnect with the people who matter most.

What it does

Memory Palace is an AI-assisted reminiscence platform for dementia support.

  • Caregivers upload meaningful photos and memories.
  • The system generates explorable 3D memory spaces from those inputs.
  • Patients enter a calm “memory lobby” and step into individual memories through different doors.
  • Inside each memory, they can interact with voice-guided reminiscence prompts while walking around the 3D world.
  • A conversational memory companion helps sustain dialogue and reflection.
  • Session and engagement data help caregivers track progress over tim

How we built it

Tech Stack:

  • Frontend: Next.js, React, TypeScript, Tailwind CSS
  • 3D rendering/UI: Three.js
  • Data/backend: Supabase (Postgres + API)
  • Conversational companion: OpenAI (GPT OSS), Web Speech API
  • Gaussian splat generation pipeline: World Labs

More about World Labs + Gaussian Splatting:

A key part of the experience is generating photoreal, navigable memory scenes. We used World Labs to transform uploaded memory imagery into Gaussian-splatted 3D worlds. Those generated assets are then linked into our memory records in Supabase and loaded in the viewer route. This allowed us to:

  • Move from static photos to spatially explorable memory environments.
  • Preserve visual detail while maintaining interactive performance.
  • Build a scalable pipeline from caregiver upload -> world generation -> immersive playback.

More about Conversational memory companion (OpenAI GPT OSS) + Web Speech API:

We integrated OpenAI (GPT OSS) to power the conversational companion and voice-guided prompts coupled with web speech API for voice recognition. The assistant is designed to:

  • Ask gentle, context-aware reminiscence questions.
  • Encourage emotional recall without overwhelming the patient.
  • Adapt tone and pacing to supportive care interactions.
  • Help caregivers scaffold future sessions from prior conversations.

Challenges we ran into;

  • Ensuring viewer stability for large splat assets and varied camera starting positions.
  • Balancing technical sophistication with simple UX for patients and caregivers.
  • Keeping clean data with multiple tables in a relational database format.
  • Troubleshooting unexpected database crashes and monitoring system health.
  • Layering conversational AI elements with world rendering in an intuitive manner.

Accomplishments that we're proud of

  • Built an end-to-end pipeline from images of memories to immersive 3D reminiscence.
  • Integrated World Labs Gaussian splatting into an interactive care experience.
  • Added voice-guided prompts and a conversational memory companion using OpenAI GPT OSS.
  • Developed an immersive 3D experience from copious doors that open to memories that can be relived once again.
  • Created a product direction that is both technically advanced and impactful for people that struggle with dementia or simply want to walk down memory lane.

What we learned

  • World-generation pipelines require strong state management and data contracts.
  • Camera defaults and scene framing are critical for the best UX experience, especially in theraputic contexts.
  • Healthcare-adjacent products demand reliability, transparency, and graceful fallbacks.
  • Conversational AI for memory care needs careful prompt design, not generic chat behavior.
  • Immersive tech only matters if the experience feels emotionally safe and intuitive.

What's next for Memory Palace

  • Detailed scene annotation allowing specific references to parts of the scene
  • Relevant objects from the scene get converted into 3D objects that are inspectable
  • Expand voice-guided reminiscence with adaptive prompt personalization per patient history

Built With

Share this project:

Updates