ClayQuest

ClayQuest turns children’s hands into stories. Kids sculpt with clay, snap a photo, and turn their creation into a narrated picture book—becoming authors, illustrators, and storytellers through play.

Comment

About the Project: ClayQuest

Team: Xiao He, Dehong Hao, Olasile Abolade

Inspiration

ClayQuest was inspired by hands-on work with children and a deep belief in the power of tactile creativity.

Xiao He is a San Francisco–based artist and children’s art educator who runs a nonprofit organization called Little Moss, where she hosts workshops inviting children to draw, sculpt, and experiment with materials such as air-dry clay. Through these experiences, she observed how naturally children invent characters and worlds when working with their hands—but also how difficult it can be to help them carry those ideas into sustained storytelling.

Xiao had long wanted to explore whether AI could be used not to replace imagination, but to support and extend it—helping children turn physical making into stories they can see, hear, and share. ClayQuest emerged from this desire to bridge tactile creation with narrative play, empowering children to author their own stories starting from what they physically create.

What It Does

ClayQuest turns children’s hands into stars.
The platform transforms children’s hand-made clay creations into narrated picture books with images and audio. By starting from something they sculpt themselves, children become authors, illustrators, and storytellers through play—rather than passive consumers of digital content.

How We Built It

We began by doing the activity ourselves. Each team member worked with real air-dry clay, creating simple sculptures and objects to understand the tactile experience from a child’s perspective. Starting with the physical process helped ground all later technical decisions.

From there, we aligned our approach with the hackathon criteria and identified the core technical needs:

  • Live camera input and image capture
  • Image-to-text interpretation
  • Story generation tied directly to the child’s creation
  • Audio narration to bring stories to life

We designed a pipeline that moves from physical object → captured image → AI-generated story → narrated picture book, integrating tools such as ElevenLabs for voice narration.

Challenges We Ran Into

One of the biggest challenges was defining the right scope and user flow. Early on, we imagined a fully live experience where children could build characters, scenes, and worlds while a camera continuously captured and updated the story in real time.

However, we quickly realized this approach would be technically heavy and overly complex for an initial version. We took a step back and simplified the experience by combining live camera usage with intentional image capture, allowing us to preserve creativity while keeping the system usable and stable.

This shift helped us focus on what mattered most: the child’s creation and sense of agency.

Accomplishments We’re Proud Of

We’re proud of how each team member contributed from their strengths:

  • Olasile (Ola) set up the overall project structure and product foundation
  • Dehong implemented core system components using Claude Code
  • Xiao Experimented with frontend techniques using Replit, and grounded the project in real art-making practices by bringing clay and pedagogical insight into the process

Together, these contributions formed a cohesive prototype that balances creativity and technology.

What We Learned

Our team comes from different cultural backgrounds and disciplines, and for all of us, this was the first time working together in person. Despite those differences, we discovered how quickly shared curiosity and purpose can bring people together.

We also learned that meaningful creative tools emerge when art, education, and technology are given equal weight—and when collaboration leaves room for listening and iteration.

What’s Next for ClayQuest

With more time, we would love to:

  • Host play-testing workshops where children sculpt with air-dry clay and create stories on the spot
  • Observe how children naturally narrate and imagine while building, and incorporate that into the storytelling flow
  • Explore real-time or semi-real-time storytelling that evolves alongside the physical making process

Ultimately, our vision is for ClayQuest to encourage children to tell stories as they build, with AI acting as a gentle interpreter and amplifier of their imagination.

Tech Stack

AI/ML Services & APIs Anthropic Claude Sonnet 4 (story generation from images) Google Gemini API (Imagen 4.0 for image generation) Freepik Mystic API ElevenLabs API (text-to-speech)

Built With

Share this project:

Updates