Family Story Night

AI-powered bedtime stories for Fire TV blending AWS Bedrock, Polly SpeechMarks Karaōke read-along, voice remote dictation, and ambient bedtime wind-down on the living room screen.

Comment

Inspiration

In millions of living rooms every evening, bedtime is a battlefield of blue-light overstimulation. Children finish dinner and immediately gravitate toward tablets or TV screens playing hyperactive, autoplay cartoons. Rather than calming young minds down for sleep, high-frequency dopamine loops leave kids wired, making lights-out stressful for both parents and children.

At the same time, I noticed that most smart TV applications treat the living room screen as a passive one-way video player. I asked myself: What if I pushed past the obvious video player model? What if the living room TV could become a warm, interactive digital hearth that brings parents and children together, fosters early childhood literacy, and physically helps kids wind down for peaceful sleep?

That vision inspired Family Story Night: a multi-modal, agentic bedtime storytelling companion crafted specifically for Amazon Fire TV and mobile web displays.

What It Does

Family Story Night transforms the 10-foot television experience into a personalized, living storybook:

  • 🎙️ Voice Control via Alexa ("Tell the TV what to do next"): Kids and parents can speak to any Alexa device in the home to dictate listener names, customize bedtime themes, or speak surprise adventure twists directly to the TV using our custom Alexa Skill webhook integration.

  • 🎙️ Personalized Voice Clones (Family Voice Studio): Enables traveling parents, deployed military guardians, or distant grandparents to record a short bedtime reading sample right in the app so children fall asleep listening to stories read in their loved one's familiar voice.

  • 📖 Personalized Co-Created Adventures: Parents enter their children's names and choose between imaginative Bedtime Adventures (Space Explorers, Enchanted Forest, Ocean Deep) or inspiring Bible & Faith Stories (Noah's Ark, David & Goliath, Daniel in the Lions' Den).

  • 🎨 Live Generative Illustrations with Amazon Bedrock: Scene by scene, Bedrock's Stability AI (stable-image-core-v1:1) generates child-friendly watercolor illustrations, displayed with cinematic Ken Burns camera panning and floating ambient stardust.

  • 🎤 Karaōke Read-Along Highlighting (AWS Polly SpeechMarks): As Amazon Polly's neural voices narrate each page, the TV highlights the spoken words in real time with a glowing amber drop-shadow, turning the 65" TV screen into a powerful literacy and vocabulary tool.

  • 🌙 Bedtime Wind-Down Routine ("Sleepy Mode"): As the chapter progresses, the TV automatically physically dims backlight contrast, shifts ambient particle motes from cool starlight to soothing candlelight amber, and plays a gentle Web Audio pentatonic lullaby chime when the story finishes.

  • 🧭 Branching Adventure Frontier: When a chapter concludes, children are presented with bedtime reflection lessons and can choose what happens next; generating continuous follow-up chapters with full character continuity.

  • 📚 Family Bookshelf & History: Stories and completed chapters are saved locally with automatic image-quota lightening so families can revisit their favorite bedtime tales anytime.

  • 📱 Universal Responsive Design: Seamlessly shifts between 10-foot living room TV displays (D-pad remote focus rings, 5% overscan safety margins) and phones/tablets for bedtime reading on the go.

How I Built It

Family Story Night combines modern web standards with deep cloud-native AWS orchestration:

1. The 10-Foot Frontend Experience (Fire TV Web App)

  • Framework: React 19 and Vite with Tailwind CSS.

  • 10-Foot TV Remote Navigation: Built from scratch to support native Fire TV remote keycodes (ArrowUp, ArrowDown, ArrowLeft, ArrowRight, Enter, BrowserBack, and media keys) with high-contrast glowing focus rings and overscan-safe margins.

  • Synchronized Karaōke Renderer: Consumes timestamped SpeechMarks from AWS Polly to calculate sub-second character slicing on story paragraphs without jitter.

  • Zero-Asset Web Audio Chimes: Synthesizes soft, dreamy 4-note pentatonic lullaby chimes (C5, E5, G5, C6) using pure AudioContext sine-wave oscillators; zero extra network bandwidth or external audio files required.

  • Responsive Layout Engine: Uses adaptive aspect-ratio containers and CSS flex containment so illustrations and story text remain crisp and unclipped across 360px phones and 4K TVs.

2. Backend Orchestration (Node.js & Express)

  • AWS SDK Bedrock Runtime:

    • Amazon Nova Pro v1:0 (primary): Structured JSON narrative generation with actionable moral takeaways and interactive branching choices. Employs continuity prompts to expand previous chapters seamlessly.
    • Anthropic Claude 3.5 Sonnet (backup/alternative): Available as fallback model for story generation.
    • Stability AI via Bedrock (stable-image-core-v1:1): Concurrent parallel visual generation pipelines with automated prompt sanitization and fallback resilience.

  • Amazon Polly Dual-Stream Synthesis: Executes parallel calls to SynthesizeSpeechCommand to simultaneously stream 48kHz Neural MP3 narration and word-level JSON SpeechMarks (SpeechMarkTypes: ["word"]).

  • Resilient In-Memory LRU & Quota Handling: Protects browser storage limits by lightened canvas image caching so families can preserve dozens of bedtime stories.

Challenges We Ran Into

  1. Synchronizing Polly Narration with UI Karaōke:

Standard text-to-speech audio outputs do not indicate when a specific word is spoken. I solved this by calling AWS Polly's SynthesizeSpeech twice in parallel: once for the neural MP3 voice stream, and once requesting SpeechMarkTypes: ["word"] JSON lines. On the frontend, coupled HTML5 Audio timeupdate events to speechmark offsets to illuminate words with sub-second accuracy.

  1. Designing for the 10-Foot Living Room UI vs. Mobile Viewports:

Building an interface that looks stunning on a 65-inch 4K TV viewed from 10 feet away while remaining fully functional on a handheld smartphone was a major design challenge. I resolved this through responsive CSS architecture: on TVs, illustrations and text sit in a panoramic side-by-side layout with overscan margins; on phones, it stacks into a vertical storybook format with uncropped artwork and touch-friendly controls.

  1. Fire TV Storage Quotas:

Fire TV web apps run in WebView containers with limited localStorage allotments. Storing multiple high-resolution base64 illustrations caused storage quota errors. I engineered an adaptive storage lightening algorithm that preserves the cover art and chapter text while gracefully condensing older page images to ensure stories never fail to save.

  1. Voice Input on Fire TV:

The Web Speech API is blocked on Fire TV due to secure context requirements and the absence of Google Play Services. Additionally, the Alexa button on the Fire TV remote is intercepted by the OS kernel. I solved this by building a custom Alexa Skill webhook that accepts voice commands from any Alexa device in the home, with zero-friction auto-pairing for single-TV households.

Accomplishments That I'm Proud Of

  • Personalized Voice Clones (Family Voice Studio): Solved a real emotional challenge for families with traveling parents, deployed military guardians, or long-distance grandparents by letting them record a short bedtime sample so children can fall asleep hearing their loved one's familiar voice.

  • Pushing Far Past the Obvious: Instead of building another basic streaming video wrapper, I built an agentic, voice-enabled, multi-modal bedtime experience that genuinely aids parenting and sleep hygiene.

  • 100% Automated Test Coverage: Built a bulletproof testing pipeline with 202 frontend test suites and 88 backend test suites; totaling 290 automated tests ensuring every user flow, remote keycode, and audio fallback behaves predictably.

  • Amazon Web App Tester Certified: Packaged into a clean production distribution (client/family-story-night.zip) fully compliant with Amazon Fire TV web app submission specifications.

What I Learned

  • 10-Foot Design Thinking: Remote controls change everything; from focus trap management and overscan padding to the critical importance of keeping font sizes legible from 10 feet away.

  • The Magic of Polly SpeechMarks: SpeechMarks are dramatically underutilized in educational and children's apps; pairing timestamp marks with visual UI glows unlocks tremendous potential for early literacy.

  • Audio Synthesis on Constrained Devices: Creating Web Audio API chimes directly in code is vastly superior to bundling MP3 assets on TV web apps; it guarantees zero latency and zero download overhead.

  • Fire TV Platform Differences: Physical device testing is essential; emulators don't accurately simulate D-pad navigation, WebView storage limits, or Alexa button interception.

What's Next for Family Story Night

  • Login with Amazon (LWA) & Device Code Pairing: Cloud-synchronize family bookshelves across multiple Fire TVs, iPads, and mobile phones.

  • Multi-Lingual Bedtime Translation: Integrating Amazon Translate with multilingual Polly Neural voices so immigrant and bilingual families can experience heritage bedtime tales in their native languages.

  • Accessibility Features: Add closed captions, voice-only navigation for blind users, and dyslexia-friendly font options.

Built With

Share this project:

Updates

Submission history