Pediatric Journey Weaver

Turning medical procedures into magical adventures.

Comment

Inspiration

MRI scans can be scary for kids. The machine is loud, the tunnel is narrow, and children are asked to stay perfectly still.

A major inspiration was Doug Dietz’s 2012 TED Talk: “Transforming healthcare for children and their families.” We were also inspired by IRM en Jeu, a French initiative that provides rocket-shaped MRI simulators for hospitals.

These programs show how powerful playful preparation can be decreasing children sedation to 80%. However, many hospitals cannot implement them because they require space, funding, and specialized staff.

So we asked ourselves:

What if the exam could feel like an adventure instead of something frightening?

That idea became Pediatric Journey Weaver — a storytelling system where the child becomes the hero of the mission.

We shared the concept with Catherine Lafaye, Senior Health Manager of the pediatric imaging department at the renowned Necker–Enfants Malades Hospital in Paris. She confirmed that while psychological preparation is important, there are currently very few interactive tools used in the preparation process. Ms. Lafaye generously shared insights and ideas that helped shape our approach.

Through these discussions, we also realized that children’s emotional experience in medical care is often under-addressed, despite how important it is.

What we built

Pediatric Journey Weaver turns the MRI procedure into an interactive story.

For example:

  • the MRI scanner becomes a submarine cockpit exploring the ocean
  • the loud sounds become sonar pings
  • staying still becomes part of the mission

The child talks directly to an AI guide who narrates the story and reacts in real time.

Instead of passively watching a video, the child can ask questions, express fears, and interact with the adventure.

How we built it

The experience is powered by Gemini Live running on Google Cloud.

We used:

  • Gemini Live (gemini-live-2.5-flash-native-audio) for real-time voice interaction
  • Vertex AI to generate illustrations during the story
  • Cloud Run + FastAPI for the backend
  • Firestore to manage sessions and story state

The AI controls the experience using tool calls such as:

  • generating illustrations
  • playing sound effects
  • advancing the story phases
  • responding if the child sounds distressed

This makes the story feel dynamic rather than scripted.

What we learned

Designing for children required several iterations. At first, the story visuals appeared through a yellow submarine porthole, as if the child was looking out from inside the vessel. During testing we realized that limiting the visuals this way made the experience feel too constrained. We eventually switched to a full-screen display with a clean white submarine inspired by the MRI machine itself, which made the connection between the story and the real procedure clearer.

One important insight was that children prioritize vivid imagery and imagination over strict realism. They naturally fill in the logical gaps themselves, so expressive visuals and atmosphere matter more than perfect accuracy.

Voice interaction also proved essential.

Kids don’t just listen — they talk to the story.

That interaction gives them something very important during a medical procedure: a sense of control and agency in a moment where they usually have very little.

Finally, we realized that the story does not have to start in the hospital. As Ms. Lafaye suggested, the experience could begin at home, for example through a QR code on the MRI appointment letter. Parents could watch the story with their child before the visit, making the hospital environment feel more familiar when they arrive.

Challenges

The biggest challenge was building a smooth multimodal experience.

Gemini Live streams voice very well, but images must be generated separately. We built a pipeline where the AI can trigger image generation through Vertex AI while the narration continues seamlessly.

Another challenge was keeping long conversations stable, which required context management and tool-call limits.

What’s next

Right now Pediatric Journey Weaver is a proof of concept, but the idea could go further.

Instead of a single story for one procedure, children could follow a longer adventure throughout their hospital stay, unlocking new chapters along the way.

At the end, the journey could even become an illustrated book the child keeps as a memory or a poster to display in their bedrooms.

Built With

Share this project:

Updates