Rawi Ai

AI-powered Arabic storytelling where children play and the system silently assesses cognitive, linguistic, and social-emotional development — turning every story into an invisible classroom.

Comment

Inspiration

In the Arab world, over 400 million people speak Arabic — yet children face a striking gap in interactive, personalized digital learning content. Most educational apps are English-first, and traditional storybooks are static: they can't adapt to a child's interests, respond to their voice, or measure what they're actually learning.

We asked: what if every bedtime story was also an invisible classroom?

The idea came from observing how children naturally learn — not through lectures, but through narrative. A child who hears a story about a character learning to share absorbs that lesson far more deeply than one who is told "sharing is good." We wanted to build a system where:

  • The child is a co-creator, not a passive consumer
  • Educational goals are embedded in the plot, never in a lecture
  • Developmental progress is measured silently through natural interactions
  • Parents and health/psychology centers get actionable, longitudinal data

Rawi was born at the intersection of AI storytelling, developmental psychology, and Arabic language preservation.

What it does

Rawi is a multimodal AI storytelling agent that generates personalized, interactive stories for children in Arabic and English — while silently assessing their developmental progress across six educational dimensions. The entire interface, story generation, narration, and assessment adapt seamlessly based on the selected language, with full RTL support for Arabic and LTR for English.

The Child Experience

Children don't just read stories — they shape them. Each story unfolds slide by slide, and children interact using:

  • Voice recordings (transcribed via Cloud Speech-to-Text)
  • Camera photos (analyzed via Cloud Vision AI)
  • Freehand drawings (interpreted and woven into the narrative)
  • Text input

Gemini 2.5 integrates every input as a central story element — a photo of a cat becomes a story companion, a drawing of a castle becomes the setting.

The Invisible Classroom

Behind every adventure lies a goal-driven story architecture spanning six developmental domains:

Domain What It Measures
Language Skills (المهارات اللغوية) Vocabulary acquisition, sentence complexity, keyword usage
Cognitive Skills (المهارات المعرفية) Problem-solving, pattern recognition, cause-and-effect reasoning
Social & Emotional (المهارات الاجتماعية والعاطفية) Empathy, emotional regulation, conflict resolution
Creativity & Expression (الإبداع والتعبير) Imaginative responses, open-ended thinking
General Knowledge (المعرفة العامة) World awareness, cultural knowledge
Motor Skills (المهارات الحركية) Drawing precision, gesture interaction

The educational goal shapes the story's conflict, character behavior, and resolution. For example, if the goal is "sharing":

  • The conflict arises from a character refusing to share
  • Characters model or struggle with sharing behavior
  • The resolution only works because the protagonist applies the skill

The child never feels tested — they experience an exciting adventure.

AI-Powered Assessment

After each story, Gemini evaluates the child's responses across all slides using a weighted scoring model. For each skill $s_i$ with weight $w_i$ and $m$ measurement types:

$$S_{skill} = \sum_{j=1}^{m} w_j \cdot v_j$$

$$S_{overall} = \frac{\sum_{i=1}^{n} w_i \cdot S_{skill_i}}{\sum_{i=1}^{n} w_i}$$

Results map to a four-tier proficiency model:

Level Score Range Meaning
Emerging (ناشئ) $0 \leq s < 25$ First exposure to the skill
Developing (متطور) $25 \leq s < 50$ Beginning to demonstrate understanding
Proficient (متمكن) $50 \leq s < 75$ Consistent application of the skill
Mastered (متقن) $75 \leq s \leq 100$ Reliable, independent use

Six measurement types feed into each assessment: keyword_usage, response_length, voice_confidence, choice_pattern, interaction_engagement, and creative_expression.

The Parent & Health Center Dashboard

Parents get real-time behavioral profiles updated live during story sessions via Firestore onSnapshot. Over multiple sessions, a longitudinal developmental profile builds up — making Rawi valuable not just for parents, but for:

  • Child psychology centers — early screening for developmental delays
  • Speech therapy clinics — tracking language progression over time
  • Schools and educators — measuring social-emotional growth across a class

Story Export & Print

Once a story is complete, children and parents can export the entire story as a beautifully formatted PDF — complete with AI-generated illustrations, narration text, and the child's name on the cover. Stories become printable keepsakes that children can hold, share with friends, or add to a physical bookshelf. The PDF is generated server-side using PDFKit with full Arabic/English text rendering and right-to-left layout support.

Bilingual Support (Arabic & English)

Rawi is fully bilingual. Parents select the story language during session setup, and the entire pipeline adapts:

  • Story generation — Gemini produces narration in the chosen language (Arabic with full tashkeel diacritics, or English)
  • TTS narration — Cloud Text-to-Speech uses native Arabic or English voices
  • UI layout — automatically switches between RTL (Arabic) and LTR (English)
  • Assessment — educational evaluation runs in the story's language
  • PDF export — rendered with correct text direction and font for each language

Three Apps, One Deployment

  • Child PWA — immersive fullscreen storytelling (installable, standalone)
  • Parent PWA — session creation, goal selection, progress dashboards
  • Admin Panel — configure templates, worlds, educational goals, AI settings

How we built it

Architecture: Single Next.js 14 application (App Router, TypeScript) deployed on Google Cloud Run (me-west1), serving three interfaces from one codebase using route groups.

AI Core: All story generation uses the @google/genai SDK. Gemini 2.5 Flash generates structured JSON output (narration text + image prompts + completion status) validated against a strict Zod schema. We implemented a two-step interleaved approach: text generation with schema validation first, then image generation using responseModalities: ['IMAGE'].

GCP Enhancement Pipeline (6 services working in parallel):

Child Input (voice/photo/drawing)
    │
    ├──► Cloud Speech-to-Text ──► Arabic transcription
    ├──► Cloud Vision AI ──────► Labels, objects, safety scores
    └──► Cloud NLP ────────────► Sentiment + entity recognition
    │
    ▼
Gemini 2.5 (enriched context) ──► Story slide (JSON)
    │
    ├──► Gemini/Imagen 3.0 ────► Storybook illustration
    ├──► Gemini tashkeel pass ─► Diacritics correction
    └──► Cloud Text-to-Speech ─► Arabic audio narration
    │
    ▼
Cloud Storage ──► Client Playback

Educational Framework: Admin-configurable goals with weighted skills and measurement rules. The goal configuration is injected into Gemini's system prompt using XML-structured context, shaping story generation at every level. Post-story assessment uses a separate Gemini call with all slides as input.

Real-time Sync: Parent dashboard uses Firestore real-time listeners to track the child's session as it happens — every decision, every behavioral tag, updated within seconds.

PWA Architecture: Two separate manifest.json files with independent service workers allow the child and parent apps to be installed as separate standalone apps from a single deployment.

Challenges we ran into

  • Arabic diacritics (tashkeel): Gemini sometimes drops diacritics on final slides or mixes languages under token pressure. We added an explicit tashkeel correction step — a separate Gemini call that adds full diacritics before TTS synthesis. This was critical because Cloud TTS pronunciation quality depends entirely on diacritics.

  • Imagen safety filters: Terms related to children trigger Imagen 3.0's safety filters aggressively. We implemented prompt sanitization (e.g., "child" → "small adventurer", "boy" → "young explorer") and retry logic with progressively simplified prompts.

  • Interleaved output token budget: Using responseModalities: ['TEXT', 'IMAGE'] in a single Gemini call caused token budget conflicts where image generation consumed all tokens. We switched to a two-step approach (text first, image second) — more reliable and allows schema validation on the text step.

  • Invisible assessment design: The hardest challenge wasn't technical — it was designing assessment moments that feel natural within a story. We created a framework where each educational skill must have $\geq 1$ dedicated "stealth observation point" per story, where the child's choices reveal their understanding without any explicit testing.

  • NEXT_PUBLIC_ on Cloud Run: Next.js inlines NEXT_PUBLIC_ environment variables at build time, not runtime. We had to inject them as ENV directives in the Dockerfile so Cloud Build bakes them into the client JavaScript.

Accomplishments that we're proud of

  • Children genuinely co-create their stories — their voice, photos, and drawings meaningfully change the narrative direction
  • The educational assessment is truly invisible — children experience an exciting adventure while the system tracks six developmental domains
  • Six Google Cloud services work together seamlessly in a single enhancement pipeline with parallel execution
  • Full bilingual support (Arabic with tashkeel + English) — RTL/LTR layout, native TTS voices, and language-aware PDF export
  • Children can export and print their stories as keepsake PDFs with illustrations and personalized covers
  • A weighted, multi-dimensional scoring model that produces clinically meaningful proficiency levels — not just "good job" badges
  • The parent dashboard provides real-time session monitoring — parents can watch their child's behavioral profile update live
  • Three installable apps (child PWA, parent PWA, admin panel) served from a single Next.js deployment

What we learned

  • Gemini's structured output is production-ready: Schema-validated JSON output from Gemini 2.5 is extremely reliable — we never had to write a custom parser or handle malformed responses
  • Split interleaved generation: Combining text and image in one Gemini call is elegant but fragile; splitting into two steps is more reliable and gives better control over each modality
  • Arabic NLP requires special care: Diacritics are essential for both TTS quality and child readability, but AI models treat them as optional — explicit enforcement is necessary
  • GCP ML APIs multiply AI value: Feeding Cloud Vision labels, STT transcriptions, and NLP sentiment back into Gemini's context produces dramatically richer, more contextual story generation than raw input alone
  • Invisible assessment > explicit testing: Children engage 3-5x more deeply when they don't know they're being evaluated. The narrative wrapper makes assessment feel like play — which is exactly how children learn best

What's next for Rawi AI

  • Longitudinal analytics dashboard — visualize developmental trends across weeks/months with percentile comparisons for health professionals
  • Therapist integration — structured reports exportable to child psychologists and speech therapists, with ICD-compatible developmental milestone mapping
  • Collaborative storytelling — multiple children co-creating a story together, enabling social skill assessment in real-time group dynamics
  • Offline mode — full PWA offline support with background sync for areas with limited connectivity
  • Adaptive difficulty — automatically adjust story complexity and assessment depth based on the child's cumulative proficiency profile using the formula:

$$D_{next} = D_{current} + \alpha \cdot (S_{avg} - \theta)$$

Built With

Share this project:

Updates

Submission history