StoryBuddy

About the project

Inspiration

Speech practice at home often feels like drills — repeat a word, get corrected, try again. Kids shut down; parents and therapists lose the data they need between sessions.

We were inspired by research showing home speech games can improve accuracy, and that child-centred, game-like design keeps kids (including autistic kids) engaged. We wanted practice that feels like storytime, not homework: a warm narrator, short story beats, gentle retries, and a progress view grown-ups can use.

That’s how StoryBuddy and Mr. Sprout were born — a sprout-headed forest sprite who reads, listens, and cheers kids on.

What it does

StoryBuddy turns speech practice into an interactive story:

A child signs in and picks a sound-focused story (or “Surprise me!”).
Mr. Sprout narrates each beat out loud, then pauses for the child’s turn.
Target words get up to three friendly tries; open questions count as engagement.
Sessions save accuracy, attempts, and simple voice features.
A therapist report gives parents a plain-language summary plus charts to share with an SLP.

Mr. Sprout’s mood tracks the moment — waving hello, talking, listening, thinking, celebrating — so the experience feels like a companion, not a quiz.

Practice accuracy for a session:

$$ \mathrm{accuracy} = \frac{\text{correct target-word attempts}}{\text{total target-word attempts}} $$

StoryBuddy supports speech-therapy practice. It does not diagnose any condition and does not replace a speech therapist.

Live demo: https://storybuddy-sagv.onrender.com/

How we built it

Backend: FastAPI + uvicorn for auth, stories, scoring, sessions, and reports
Voice: ElevenLabs for TTS (Mr. Sprout’s voice), STT (hearing answers), and story sound effects
LLM: Groq for warm reactions and the parent/therapist report
Frontend: Custom child UI + landing page; Mr. Sprout as a Rive mascot (mouthOpen driven by narration loudness)
Data: SQLAlchemy (SQLite locally / Postgres on deploy) for users and session history
Deploy: Docker + Render at storybuddy-sagv.onrender.com

Challenges we ran into

Keeping the loop kid-paced — narration $\rightarrow$ listen $\rightarrow$ score $\rightarrow$ reaction — without dead air or scary “wrong” moments
Syncing Mr. Sprout with audio (Rive artboards + mouth motion) so talking looked alive
Writing honest reports without overclaiming — metrics as conversation starters, never a diagnosis
Shipping reliably: JSON $\rightarrow$ SQL, containers, env secrets, and a public demo that stays up

Accomplishments that we're proud of

A full story loop where kids talk to Mr. Sprout and get kind, specific feedback
A living Rive narrator whose mood matches listening, talking, and celebrating
Session history that becomes a calm, shareable therapist report
A live deployed demo families and judges can try without setup

What we learned

Warmth (latency, retry copy, mascot mood) matters as much as model quality
Voice APIs are product UX — TTS pacing, STT noise, and typed fallback must feel like one conversation
Kids need stars; adults need clear numbers they can take to a therapist
Deploy, data persistence, and safety wording are part of building something real

What's next for StoryBuddy

A bigger library of stories across more speech sounds
Animated, interactive video stories that respond as the child talks
A therapist-facing app to review each child’s practice history in one place
Richer progress insights over time — still as practice support, never as diagnosis