SyncTales

• 

  Synctales - stories that keep your people close.

Inspiration

SyncTales comes from my own situation as a father of two young sons (Virgil 5, and Raphael 3). Living and working apart from them means I miss crucial moments like story-time and the small learning opportunities, connection, and confidence that can grow around it. So, I built something I wanted for myself, and something I thought a lot of other people could benefit from.

What it does

SyncTales is a family-led, AI-assisted shared-reading companion. An adult sets the starting context (age, reading ability, etc.), then reads along with the child using live voice followingAt the end mini games are generated for the child (rooted in science of learning/reading research) and a few optional story-grounded moments around sounds, vocabulary, expression, and comprehension are displayed throughout the narrative to keep the child engaged.

After a story a parent insight is created that covers: what was practised, what may be worth revisiting, and one small next-read ritual. It intentionally avoids child scores, open chatbot interaction, and an AI voice competing with the parent or child. The goal is to help the parent (and child), not replace them.

How we built it

SyncTales is a React, TypeScript, and Vite web app deployed on Vercel with Supabase for the family, session, story, consent, and derived-insight data model.

I built it through a human-directed collaboration with Codex (for all coding work via VSCode Plug-In) and GPT-5.6 (Terra for 80% of coding, Sol for difficult aspects and discussion). GPT-5.6 is a real runtime component: before a story begins, it creates one schema-constrained plan for the entire read from approved story variants and bounded evidence. It selects a reader-matched rendition, a small set of optional AI Moments, and later-read Word Twists while preserving the story’s plot, characters, page order, and safety meaning.

Live reading uses transient OpenAI Realtime transcription with a deterministic, page-local sequence-alignment engine. GPT-5.6 never controls the visual reading marker. A short optional retell is transcribed transiently (no unnecessary data is stored) and analyzed with GPT-5.6 into structured, parent-facing observations rather than an open-ended chat response.

I used NanoBanana for original visual asset generation (using GPT 5.6 to help refine prompts)—backgrounds, characters, and props—while keeping the reading text (which is grounded in open-source classic stories to prevent hallucinations), interaction UI, learning metadata, and product logic controlled in the app.

Challenges we ran into

The hardest problem was sequence alignment for the live word highlight. A transcript is not the same thing as knowing where a reader is on a page: repeated words, restarts, pauses, mispronunciations, partial sentences, and delayed transcription (all events common with reading at a young age) events can all make a naïve “find the last word” approach jump unpredictably. I spent around 16 hours trying to fix this (it was a blast).

I solved this (with the help of Sol) with a stable token map for each displayed page, separate confirmed/live/displayed progress states, contextual phrase matching, a confidence buffer for backward jumps, and rejection of ambiguous duplicate phrases instead of guessing. It also surfaced a useful conceptual parallel with LLM attention: ambiguous tokens need surrounding context to resolve meaning—but the child-facing marker remains deterministic, not an LLM decision.

Accomplishments that we're proud of

I'm proud that SyncTales is not just a basic “AI that reads to your child.” It is a complete, working loop that helps preserve family connection while making learning support more useful: shared story, bounded AI Moment, optional retell, derived Insight, and one practical next step.

I'm also proud of the restraint in the implementation. GPT-5.6 adapts the experience dynamically overtime (rooted in the child progress), but cannot rewrite the story freely, take control of the reading cursor, invent an open child chat response, or turn one reading moment into a permanent label. The adult-child relationship remains central.

What I learned

This build pushed me far beyond the initial product idea. I learned about COPPA and the responsibility of handling child voice data; the real limitations of browser speech recognition and general transcription models for developing readers; and the engineering depth behind sequence alignment.

I also learned that an education product needs to be grounded in learning/reading science, not just good UI. Research around phonological awareness, synthetic phonics, vocabulary, prosody, comprehension, and retell fundamentally influenced the product direction.

What's next for SyncTales

Next, I will (try to) turn this prototype into a production-ready web product: complete the authenticated family flow, pedagogically sound educational scaffolding to support children from learning sounds to independent reading, strengthen consent and privacy controls, validate the speech path with representative child voices, and test whether families return for the next-read ritual.

If the web product proves demand and product-market fit, the next platform step is a mobile app. Longer term, I want to build relationships with independent illustrators, animators and writers (eventually publishers) to enhance our content pipeline with real stories, provide a wide range of narratives that serve kids aged 2-12 from diverse cultural background and I want to support remote co-reading through synchronized book state and privacy-conscious audio/video, so parents, grandparents, extended-family and caregivers can stay present in a child’s reading life even when they are apart.

Built With

• accessibility
• ai
• codex
• edtech
• education
• ehri'sphasesofwordreading
• gpt-5.6
• javascript
• nanobanana
• openaiapi
• openairealtimeapi
• react
• revisitteachguidepracticeapply
• scienceofreading
• supabase
• typescript
• vercel
• vite
• webrtc