Inspiration
What we realized was that we often didn't learn about our family's lives unless we explicitly asked. Every family has stories that live in only one or two people's heads. Photos get saved, but the story behind them usually doesn't. Even when the story is written down, it is only one person's version.
We wanted something a grandparent and a seven-year-old would both pick up. A pop-up book is enjoyable across all generations and engaging. So we asked: what if everyone in a family could add their memories, and each one became a page in a handmade pop-up book that plays the memory back?
What it does
Stitch is a pop-up storybook of your family, made from everyone's memories.
One person starts a family and shares an invite link. Everyone claims their place on the family tree, and relatives who will never sign up (great-grandparents, young kids) can still be added so others can tell stories about them.
To add a memory, answer a question like "How did you meet Mom?", give a year, and optionally add photos. You can: 1) type it out in our web app 2) tell it out loud on a phone call with our AI agent and review the transcript before it is saved. 3) text with the AI agent
After submitting, watch your memory become a page. Stitch turns the memory into an animated spread in a 3D book. The scenery rises off the page, paper puppets of the people who were there walk, hug and hand each other things, the camera glides through the scene, and the story is narrated in the teller's own words. Every person has a book. Click anyone on the family tree and their book opens, with their memories in order by year. A memory belongs to everyone in it, so Dad's "how I met your mom" page is also in Mom's book. Like a real pop-up book, spreads have pull tabs that move parts of the scene, and clicking a character or object explains why it matters to the story.
A second view shows the whole family as stitched threads along a timeline, one color per person, with knots where people share a moment.
How we built it
A bulk of the work was the compiler. The director does not animate anything. It writes a screenplay in a small vocabulary ("dad walks to the steps", "mom drops the hymn books", "dad hands them to mom"). A compiler written in plain Python, with no model calls, turns that into keyframes. It tracks who is holding what, rejects impossible actions with exact errors, solves arm reaches with two-bone inverse kinematics, and shifts actions to land on the narrator's words. The model writes the story beats, and code makes them move.
Stitch’s studio is one create_deep_agent graph running Claude. Our langchain deepagent plans with write_todos, which streams to the family as a live progress feed, then delegates to eight sub-agents (interviewer, storyboarder, prop master, set designer, director, narrator, checker, linker) that hand off drafts through the agent filesystem. Skills teach storyboarding and directing; per-family long-term memory keeps characters consistent across books. Human-in-the-loop interrupts pause for follow-up questions and prop review. The director’s screenplay goes through a write-compile-critique loop with a deterministic compiler and a vision checker. MongoDB checkpoints make every run resumable.
The loop. After compiling, the checker renders frames of the real scene in a headless browser, looks at them, and sends problems back to the director. Compile errors go back the same way.
A big challenge was making the stage. The book is a react-three-fiber scene in a Next.js app: a 3D book with pages that curl as they turn, hinged paper layers that rise from the fold, a shared puppet rig, props with their own parts and states, and a camera track. The stage only plays the compiled timeline, so the agent, the checker and the browser always agree on what a scene looks like.
Props and scenery are drawn by an image model in a locked cut-paper style, cut out, and rigged automatically. The family's saved props and places are reused so the green sedan looks the same in every memory. Narration uses text-to-speech with word timings, with the browser's own speech as a fallback.
For database we used MongoDB and LangGraph. A FastAPI server on MongoDB Atlas: the family tree through $graphLookup, photos, cutouts and audio in GridFS, vector search for "ask the family" and for matching saved props, and LangGraph checkpoints so a run can pause for the teller and resume. Every family is private. Every route checks membership, every query is scoped to one family, and the agent's tools are bound to the family that started the run.
A separate bridge lets a relative tell a story by phone through an ElevenLabs voice agent, with signed webhooks, a pairing code, and a private draft the teller reviews first.
Challenges we ran into
Getting a language model to animate. Models are poor at coordinates and timing. Letting the model write a readable screenplay and having code do the motion was the decision that made scenes work, and exact compile errors were what let the agent fix its own mistakes.
Making paper look like paper was also really difficult. A convincing page turn, pop-ups that rise on hinges, soft shadows and a steady frame rate took far more iterations than any single feature.
A big problem was speed. A full run with art, narration and checking takes minutes. We stream the agent's plan and progress to the page so the wait is something to watch, and we prepared scenes for the live demo. One way we did this was after talking with the LangChain team, we changed our instructions and context to free up the deep agent to work more efficiently.
Four people and many agents in one repo. Our claim system once blocked the whole team because a claim message had a sentence after its file list and the hook read every word as a path. We tightened the format and kept going.
Accomplishments that we're proud of
- A memory typed into a box comes out as an animated pop-up scene with puppets, props, camera moves and narration.
- Two people's tellings of the same moment play on the same scene, each in their own words.
- An agent that checks its own work by looking at rendered frames.
- A book that people want to touch: page curls, pull tabs, click-to-explain.
What we learned
- Creating a generative system that doesn't purely rely on LLM's is very difficult as displayed with our compiler. It takes a lot of moving parts and requires you to rethink how we process input. -Agree on the data formats first. Writing one scene by hand as both a screenplay and a timeline let the stage, the compiler and the agent be built at the same time. -An agent improves quickly when its errors are exact and it can see the result of its own work. Coordination is its own engineering problem. Clear ownership, add-only contracts and a separate reviewer kept main demoable all weekend.
What's next for Stitch
-Hosting it so real families can use it, starting with our own. -Faster runs, with more of the art generated in parallel and reused. -Printing a family's book as a real, physical pop-up book. -Richer scenes: props with more moving parts and more ways for puppets to interact.
Built With
- claude
- codex
- css
- deepagent
- elevenlabs
- langchain
- mongodb
- photon2.0
- python
- three.js
- typescript