Living Photos

🌅 Living Photos

Step inside a memory you can walk through.

Living Photos turns a single photograph into an immersive, walkable 3D memory. Upload an old photo of a place that mattered to you, and Living Photos reconstructs it as a spatial scene you can explore — complete with ambient sound and the voice of someone connected to that memory.

Instead of simply looking at a photograph, we wanted to ask:

What if you could step inside it?

💡 Inspiration

Photographs preserve what a moment looked like, but much of a memory disappears with time: the space around the camera, the sounds of the environment, and especially the voices of the people who were there.

Living Photos started from the idea that generative AI could help photographs become something more than static records.

Our demo follows Walter, 72, who discovers an old photograph of his childhood home. Rather than simply viewing it, he enters the memory, walks through the reconstructed environment, and hears his father's voice again.

That became the core idea behind Living Photos:

Turn photographs from something you look at into places you can experience.

🛠 How we built it

Living Photos combines several generative and web technologies into one end-to-end pipeline.

A user begins by uploading a photograph.

World Labs Marble transforms the photograph into a Gaussian-splat 3D environment. FAL Hunyuan3D generates additional object-level 3D meshes, while ElevenLabs powers voice cloning, narration, ambient sound effects, and generated audio.

The resulting environment is rendered using React Three Fiber + Three.js, allowing users to move through their memory using familiar 3D controls.

Because generating a complete memory can take several minutes, we built the workflow as a durable 7-step Inngest pipeline rather than relying on a single server request.

The application itself is built with Next.js 15, React 19, TypeScript, Tailwind CSS, Drizzle ORM, Neon Postgres, Vercel Blob, and Stripe.

⚡ Mock-first architecture

One of the engineering ideas we're especially excited about is our mock-first architecture.

Every paid external service sits behind a common TypeScript interface with two implementations:

MockAdapter — deterministic, instant, and requires no credentials
RealAdapter — connects to the production API

This means the entire Living Photos pipeline can run locally without API keys or paid API calls.

It also lets our CI test the same contracts used by the real services without repeatedly spending credits. When the appropriate API key is added, the corresponding real-service contract tests automatically activate.

The mock pipeline completes in seconds, while a real Living Photo takes approximately 4–5 minutes to generate.

🎙 Building voice memories responsibly

Voice is one of the most emotionally powerful parts of Living Photos, but it also creates significant potential for misuse.

We therefore built a consent system directly into the product.

Before creating a voice clone, the user must complete a live-attestation flow by speaking a randomly generated phrase. We also added protections against public-figure impersonation, along with abuse reporting and rate limiting.

The goal was to treat consent and safety as part of the architecture rather than something added after the demo was finished.

🚧 Challenges we faced

The hardest challenge was coordinating several expensive and asynchronous AI systems while still creating an experience that felt immediate.

3D reconstruction, mesh generation, voice synthesis, sound generation, storage, and payments all have different latency and failure characteristics.

That pushed us toward the adapter architecture and durable background pipeline.

Mobile 3D rendering created another challenge. Large Gaussian splats can overwhelm mobile browsers, particularly Safari, so Living Photos detects mobile devices and serves a lighter rendering tier.

We also had to think carefully about voice cloning. A product built around memories needs strong safeguards against impersonation and unauthorized voice use, so consent verification became a core part of the product.

📚 What we learned

Living Photos taught us that building with generative AI is increasingly less about calling one model and more about orchestrating specialized models into a reliable product experience.

Image-to-3D generation, voice synthesis, sound generation, payments, storage, and interactive rendering all need to work together.

We also learned that emotional AI experiences require more than technical capability. Privacy, consent, reliability, and user trust need to be designed alongside the generative pipeline.

✨ What's next

We want Living Photos to move beyond individual photographs toward richer collections of memories.

Imagine digitizing an old family album and gradually transforming it into a collection of places you can revisit — childhood homes, weddings, vacations, neighborhoods, and moments involving people whose voices you want to preserve.

A photograph captures an instant.

Living Photos tries to preserve the world around it.