Memora

TL;DR: Memora turns personal stories into immersive memories you can revisit, share and build on together. Our long-term vision is Pokémon Go for memories: a worldwide map where visiting a place unlocks stories people have chosen to share there.

Inspiration

Memora turns personal stories into immersive places people can explore together.

We wanted to make it easier to share the experiences that shaped us: a childhood neighborhood, a family celebration, an afternoon with someone we love. Photographs, recordings and conversations each preserve part of that experience. We wondered what it would feel like to bring them into a shared space, where the storyteller could guide someone through it.

A memory also holds multiple perspectives. A grandmother and her grandchild might remember the same afternoon from different heights, with different details and different emotions. Memora gives both perspectives a place to exist, with each contribution attributed to the person who shared it.

This has possibilities across family history, cultural preservation, education and connection across distance. Reminiscence for families living with dementia is one particularly meaningful use case. We designed the experience to be patient and person-led, without quizzing people or treating uncertainty as a mistake.

What it does

Memora is a shared memory experience that runs in the browser on a laptop, phone or Meta Quest.

Start by telling a story. A voice companion welcomes the person and listens as they describe a place or moment. They can also share a photo or video, or invite someone to contribute. The companion supports English, Spanish and French, with automatic or selected language settings.
Step into the experience. The app uses prepared environments or scene planning to illustrate the story. Supported details can appear as people describe them. Our demonstration uses a cached meadow, allowing the conversation and interactive objects to develop without waiting for a new world generation.
Experience another perspective. Participants share the same place while keeping their own viewpoint. Switching between grandparent and grandchild perspectives changes viewing height, position and the stories associated with those eyes. Each participant can explore independently.
Bring your photos into the story. Upload a family photo directly, or use Apify to import photos and captions from public Instagram posts you select. Review the source, add a note, and attach it to a memory object or chosen perspective. Connecting imported photos to full environment generation is our next step.
Discover context inside the scene. Selecting a supported kite, flower or tree opens a short explanation retrieved by our custom Apify Actor, with its publisher and source link. Separate workflows support historical photographs and place references. Families review retrieved material before attaching it or placing an optional model.
Keep the visit. Snapshots, words and consented recordings become moments people can revisit. A map-based album connects stories to places. The current world-memory experience includes clearly labeled fictional examples that demonstrate how discovering other people’s stories could work.

The experience preserves a distinction between what someone said, what another person contributed and what the software illustrated.

How we built it

Voice and AI: Cartesia Ink-2 transcribes speech, and Cartesia Sonic voices the companion. Claude, through the Anthropic SDK, interprets narration into conversational replies and structured scene suggestions. Application code validates those suggestions and controls the actions they can trigger.
Shared conversation: We built coordination for multiple participants’ microphones, attributed contributions and a companion whose responses reach both devices. The conversation logic accounts for pauses and duplicate speech heard across microphones.
Real-time collaboration: SpacetimeDB holds the collaborative layer: participant presence, perspectives, stories, conversation state and memory handoffs. Transactional reducers and subscriptions keep that shared state connected through an authenticated Node WebSocket gateway.
Persistent application state: Convex manages rooms, permissions, media, shared scene plans, reviewed objects, saved moments and background jobs.
Worlds and interaction: World Labs Marble supplies generated surroundings, rendered with Spark. Separate GLB models provide interactive objects. We used Blender for props and rigged MakeHuman characters for participants, with Rapier supporting collision in the meadow.
Client and VR: React, TypeScript, Vite, Three.js, React Three Fiber and WebXR power the main experience. The repository also includes a separate headset client using Meta’s Immersive Web SDK. The world map uses d3-geo and TopoJSON.
Apify: Our custom Source Atlas Actor retrieves and normalizes public references across historical archives, Maps, selected Instagram posts and educational object information. It retains attribution and uses bounded requests, saved run IDs and caching. Optional image-to-3D generation uses Tripo after separate consent.
Build process and testing: Codex helped us implement, debug and iterate on integrations. Playwright exercised browser and multi-participant flows; Vitest and convex-test checked application and backend behavior. Quest emulation helped us inspect stereo rendering and controller interactions while keeping physical-headset validation separate.

Sponsor tracks

Apify — Best Use and Best Actor

What we built: Source Atlas, a custom Actor for historical archives, place references, selected public Instagram posts and educational object information, with attribution, bounded requests and caching.
Why it stands out: Retrieved material becomes a reviewed conversation cue inside an immersive experience. Source cards work in VR, and our Maps-to-reviewed-object flow was verified with real Apify retrieval.

SpacetimeDB — Real-time collaboration

What we built: The core real-time backend for participant presence, perspective-specific stories, shared conversation and memory handoffs, using transactional reducers and live subscriptions.
Why it stands out: People contribute to one shared place while retaining independent viewpoints and attribution. SpacetimeDB connects the participants, their perspectives and the evolving conversation.

Meta — Bringing People Closer Together with AI

What we built: AI-supported storytelling on Meta Quest, phone and desktop, combining a voice companion, illustrated environments, shared presence and perspective switching.
Why it stands out: Perspective-taking becomes tangible: experience a story from another person’s height and position, hear their contributions, and add your own. AI supports an activity people do together, with applications across family history, culture and dementia-focused reminiscence.

Cartesia — Best Use of Cartesia

Cartesia powers Memora’s core voice interface: Ink-2 transcribes the storyteller, and Sonic speaks the companion’s replies. Combined with patient turn-taking, captions and multilingual support, voice lets people shape a memory through conversation. This makes the experience accessible across generations and keeps the storyteller in control of the pace.

Convex — Best Use of Convex

Convex provides Memora’s persistent, reactive foundation: rooms, permissions, consent, media, scene plans and saved moments. Server-side functions coordinate source imports and generation jobs while keeping provider credentials private. It connects the complete journey—from reviewing a reference and placing an object to preserving the visit—so AI-assisted experiences become lasting, revisitable memories.

Challenges we ran into

Combining generated scenery with interaction. A visually detailed splat does not automatically provide individually editable objects. We had to align separate models, collision geometry and viewpoints with the surroundings.
Sharing a place without forcing a shared viewpoint. Participants need consistent people, objects and stories while retaining control over whose perspective they explore. This required separating shared state from local viewing state.
Making voice feel patient. The companion needs to allow pauses and let people talk to each other. We worked on turn-taking, interruption and duplicate-microphone handling so it could participate without dominating.
Keeping references separate from recollection. An archive photograph or generated prop can start a conversation, but it cannot establish what happened in someone’s life. Attribution, review and clear sample labels became part of the interaction design.
Managing asynchronous services. World generation and retrieval introduce delays and failures. We used prepared worlds, durable jobs, caching and bounded requests to keep the main visit independent of those operations.

Accomplishments that we’re proud of

We connected a complete journey: conversation, entry into a memory, shared perspectives, in-world discovery and a saved visit.
We built collaboration around individual viewpoints and attributed contributions.
We verified a real Apify Maps retrieval through family review, optional object placement and a source card displayed in stereo VR.
We implemented native 3D information cards so discovery remains available inside immersive VR.
We brought voice, real-time collaboration and source retrieval into one experience accessible from multiple device types.

What we learned

The most meaningful interaction was often a small contribution: someone noticing an object, sharing another detail or hearing a familiar voice.

AI worked best when it helped people express and explore their own stories. Structured suggestions, explicit review and deterministic validation gave us clearer boundaries around what the system could change.

We also learned that immersion depends on consistency: the right eye height, an object within reach, a responsive conversation and another person standing where they should be all matter.

Accessibility and consent shaped the product itself—from patient pauses and readable captions to recording permissions and control over sharing.

What’s next for Memora

Connect approved photos to environment creation. We want to link the existing photo-import workflow to world generation. Apify would retrieve the public Instagram posts people explicitly select and relevant place context. After review, those photos could inspire a shared environment through World Labs—a childhood garden, wedding venue or neighborhood that people can explore and enrich together.
Make discovery conversational. People should be able to ask about a supported object or request a reference through the companion. We also want saved memories to retain the complete connection between sources, reviewed illustrations and individual contributions.
Expand shared storytelling. We plan to improve multi-speaker conversations, multilingual experiences and persistent sharing across devices, alongside physical Quest testing and sessions with consenting families and care partners.
Grow into a worldwide memory map. Over time, people could discover stories others choose to publish at real places. Visiting New York might unlock memories tied to its neighborhoods—something like Pokémon Go for human memories. This would extend Memora into local history, cultural archives and immersive education.

Our vision is to give the stories that shaped us a place people can return to—and keep building together.