Children's Story AI

• 

  story2

• 

  story3

• 

  story4

• 

  cloud

• 

  story6

• 

  story1

• 

  story5

Inspiration

Bedtime stories are a treasured ritual, but as parents, I eventually run out of fresh ideas. Modern AI chatbots are great at generating text, but children don't engage with walls of text—they need vibrant visuals, cheerful sounds, and the magic of seeing themselves (or their favorite toys) right there in the adventure.

I was deeply inspired by the "Creative Storyteller" prompt for this hackathon. I wanted to build an agent that acts incredibly fast, thinking like a multimodal creative director that doesn't just reply with text, but instantly weaves an interleaved tapestry of customized illustrations, audio, and storytelling into a fluid comic book.

What it does

Children's Story AI generates a personalized, 6-page interactive comic book based on user input. Users enter character names, select an adventure theme, and importantly, can upload a photo of a subject (like a child or a favorite teddy bear). The agent concurrently generates a structured narrative, text-to-speech audio for every panel, and high-quality illustrations that strictly maintain the visual consistency of the uploaded photo.

How I built it

The application was built entirely using React, TypeScript, and Vite on the frontend, using the Google GenAI SDK (@google/genai) to securely interface with Google Cloud's Vertex AI endpoints.

My agent pipeline works in three distinct multimodal phases:

1. Vision Phase (gemini-3-flash-preview): I pass the user's uploaded photo to the model, asking it to output a concise "Visual Blueprint" (e.g., a fluffy orange dog with a blue collar).
2. Narrative Phase (gemini-3-flash-preview): The model consumes the blueprint, characters, and topic to output a strict 6-page structured JSON containing narration, dialogue, image generation prompts, and UI hotspot configurations.
3. Concurrent Asset Generation: Finally, the app fires off concurrent asynchronous requests per page:
   • Visuals (gemini-3.1-flash-image-preview): Generates the watercolor-and-ink panel illustrations using the scene prompts and the visual blueprint.
   • Audio (gemini-2.5-flash-preview-tts): Generates base64 audio data using the 'Kore' voice to cheerfully narrate the text.

Challenges I ran into

My biggest challenge was maintaining visual consistency across 6 distinct AI-generated images. Early iterations produced "space adventures" where the main character changed species in every panel!

I solved this through a "Visual Blueprint" architecture—using a preliminary Gemini vision prompt to create a rigid textual description of the character, which is aggressively appended to every subsequent gemini-3.1-flash-image-preview call.

Another challenge was performance. For a 6-page comic, my architecture must execute exactly 14 distinct API interactions (1 Vision, 1 JSON, 6 Images, 6 Audio). Managing this massive number of concurrent API calls required careful frontend state management to ensure a rapid, non-blocking flow that keeps the user experience seamless.

What I learned

I learned that the true power of Gemini lies not just in text generation, but in its mixed-modality capabilities. Orchestrating gemini-3-flash, gemini-3.1-flash-image, and gemini-2.5-flash-tts in unison proved that I can move far beyond simple text-in/text-out chatbots. I also learned how incredibly fast modern image generation models have become, allowing me to generate 6 images entirely on-the-fly without relying on pre-rendered assets.

What's next for Children's Story AI

In the future, I want to allow the Live Agent to listen via microphone! A child could interrupt the story midway (e.g., "No, I want a dragon to show up!"), and the agent would use the Gemini Live API to rapidly discard the upcoming JSON pages and dynamically re-render the story structure, audio, and images on the fly!

Built With

• ai-studio
• antigravity
• gemini-api
• google-cloud
• react
• typescript
• vite