Paper Tales

AI-powered pipeline that transforms academic research papers into illustrated, narrated stories for young readers.

Comment

Inspiration

Research papers hold the keys to humanity's most important discoveries — vaccines, climate models, quantum computing — but they're locked behind dense jargon, complex equations, and 30-page PDFs. A curious 8-year-old who asks "How do black holes work?" deserves a real answer drawn from real science, not a dumbed-down Wikipedia summary.

PaperTales bridges that gap. It takes any research paper and transforms it into a beautifully illustrated, narrated story tailored to the child's age — making cutting-edge science accessible to every young reader.

What it does

PaperTales is a 9-agent AI pipeline built with Google ADK that converts academic research papers into multimodal stories:

  • Input: Paste a paper URL from 8 supported archives (arXiv, bioRxiv, medRxiv, ChemRxiv, SSRN, EarthArXiv, PsyArXiv, OSF) — or just ask a question like "What is a transformer?" and PaperTales finds the best matching paper automatically via Gemini-powered search.
  • Processing: The paper flows through 9 specialized agents — parsing, concept extraction, language simplification, narrative design, input gating, interleaved story+illustration generation, audio narration, fact-checking, and assembly.
  • Output: A complete illustrated story with:
    • Interleaved text and images generated in a single Gemini call (mandatory for Creative Storyteller track)
    • Voice narration for every scene via Google Cloud TTS
    • Fact-check score using embedding-based semantic verification against the source paper
    • Interactive glossary with tooltips for scientific terms
    • Community voting with automatic regeneration at quality thresholds

Users choose from 3 age groups (6–9, 10–13, 14–17) and 4 story styles (fairy tale, adventure, sci-fi, comic book). Each combination produces a unique story with age-appropriate vocabulary, narrative complexity, and illustration style.

How we built it

Agent Framework — Google ADK:

PaperTales uses ADK's SequentialAgent as the backbone, with 9 specialized agents:

  1. Paper Parser (BaseAgent) — Downloads and structures the PDF using PyMuPDF/pdfplumber. Skips the LLM call entirely when the paper is cached.
  2. Concept Extractor (LlmAgent) — Identifies 3–5 core science anchors and classifies the paper into one of 14 fields of study.
  3. Language Simplifier (LlmAgent) — Reduces vocabulary and complexity for the target age group, verified with Flesch-Kincaid readability scoring.
  4. Narrative Designer (LlmAgent) — Creates a plot outline with characters, conflict, and scene structure using curated story templates.
  5. Narrative Gate (BaseAgent) — Truncates input to 100K chars and validates science anchor coverage before the Story Illustrator.
  6. Story Illustrator (LlmAgent, gemini-2.5-flash-image) — The core creative agent. Writes the full story with interleaved text + images in a single call using response_modalities=["TEXT", "IMAGE"].
  7. Audio Narrator (BaseAgent) — Generates age-appropriate voice narration for each scene using Gemini TTS.
  8. Fact Checker (LlmAgent) — Verifies story accuracy against the source paper using gemini-embedding-001 for semantic similarity.
  9. Story Assembler (LlmAgent) — Packages the final JSON and persists to Firestore + Cloud Storage.

Agents 7 and 8 run in parallel via ADK's ParallelAgent — they share no read/write dependencies, saving 30–60 seconds per pipeline run.

Every agent communicates through shared session state with documented state contracts (e.g., Agent 6 writes generated_story, which Agents 7, 8, and 9 all read).

GCP Stack:

  • Cloud Run — Backend (FastAPI, 4Gi RAM, 2 CPU, 600s timeout)
  • Firebase Hosting — Frontend (React + TypeScript)
  • Firebase Auth — Google, email, and anonymous sign-in
  • Cloud Firestore — Story metadata, voting, leaderboard
  • Cloud Storage — Images, audio files, full story JSON (workaround for Firestore's 1MB document limit)
  • Cloud Build — Full CI/CD pipeline (cloudbuild.yaml)
  • Secret Manager — API key storage

Smart Caching:

Story IDs are deterministic — sha256(paper_id + age_group + style) — so the same request returns a cached result instantly without re-running the pipeline.

Challenges we ran into

  • Image generation model limitations: gemini-2.5-flash-image does not support function calling, so the Story Illustrator cannot use tools. All scene text and illustrations must be generated in a single monolithic call, which takes 90–180 seconds and accounts for 40–60% of total pipeline time.
  • ADK single-parent constraint: ADK agents can only belong to one parent agent. We couldn't reuse agent instances across different SequentialAgent pipelines and had to design around this limitation.
  • Token budget management: The interleaved generation model has a 32K input token limit. We added a Narrative Gate agent (Agent 5) that truncates input to 100K chars and validates science anchor coverage before passing to the Story Illustrator.
  • Rate limits: Free-tier Gemini accounts are limited to ~30 image generation requests/day. We implemented deterministic caching, retry logic with exponential backoff (5 attempts, 2–60s), and graceful degradation for TTS quotas.
  • Firestore 1MB limit: Complete stories with base64 images exceed Firestore's document size limit. We implemented a hybrid storage pattern — metadata in Firestore, full story JSON in Cloud Storage.

Accomplishments that we're proud of

  • End-to-end multimodal output: Text + images + audio + fact-check score, all from a single paper URL
  • 315+ automated tests covering every agent, tool, API endpoint, and service — with zero skips
  • Embedding-based fact-checking: Not just keyword matching — we use gemini-embedding-001 to compute semantic similarity between story claims and source paper chunks
  • Full CI/CD: cloudbuild.yaml builds and deploys both backend (Cloud Run) and frontend (Firebase Hosting) in a single pipeline — infrastructure as code
  • Natural language search: Users don't need a paper URL — they can ask a question and PaperTales finds the best matching arXiv paper using Gemini-powered query refinement
  • Community features: Voting, leaderboard by field of study, and automatic story regeneration when quality drops below threshold

What we learned

  • Interleaved generation coherence: Getting Gemini to maintain character consistency across 5–6 illustrations in a single call requires detailed visual prompts in the narrative design — we learned to describe character appearances in the scene structure, not just the story prompt.
  • State contracts matter: With 9 agents passing data through shared state, documenting exactly which keys each agent reads and writes prevented cascading bugs.
  • ParallelAgent is a free win: Audio narration and fact-checking are completely independent — running them in parallel saved 30–60 seconds with zero code complexity.
  • Deterministic caching is essential: Without it, every page refresh would trigger a 3–8 minute pipeline run. The sha256(paper_id:age:style) pattern made the app feel instant for repeat requests.
  • ADK is production-ready: Despite being a new framework, ADK's abstractions (SequentialAgent, ParallelAgent, FunctionTool, state management) handled our 9-agent pipeline without issues.

What's next for PaperTales

  • Live streaming: Stream story generation progress in real-time using Server-Sent Events
  • Voice dialogue: Let children ask follow-up questions about the story using Gemini's live voice API
  • Multi-language: Generate stories in the child's native language
  • Classroom mode: Teachers can assign papers and track student engagement
  • Accessibility: Screen reader support, dyslexia-friendly fonts, high-contrast mode

Built With

  • cloud-build
  • cloud-firestore
  • cloud-run
  • cloud-storage
  • cloud-tts
  • fastapi
  • firebase-auth
  • firebase-hosting
  • gemini
  • gemini-2.5-flash
  • gemini-flash-image
  • google-adk
  • python
  • react
  • typescript
Share this project:

Updates

Submission history