Inspiration
We grew up watching cartoons like Dora ask a question, pause for an answer, and then say, “That’s right!” But what if you got it wrong? The episode kept going, and you never got the explanation you needed.
We wanted to make that interaction real. What if Dora could hear your answer, recognize when you were struggling, and help you understand before continuing the adventure?
That question became a bigger idea: building a learning experience around the cartoons kids already enjoy, with a structured curriculum behind the scenes and a dashboard where parents can follow along.
What it does
Our project turns prepared cartoon moments into interactive learning conversations. When Dora asks a question, the child answers aloud. The system interprets their answer and generates a response that fits the scene. If they get it wrong, Dora explains and gives them another question to practice before the episode continues.
Behind those conversations, lessons connect to specific skills, including counting, shapes, vocabulary, and remembering story events. Parents can use a dashboard to review answers, see where help was provided, explore progress by skill, and manage learning preferences.
For the child, learning fits into the adventure. For the parent, those interactions become something they can see and build on.
How we built it
A major part of the project was training a voice model on Dora’s audio so she could speak new, dynamically generated dialogue in a recognizable voice. We gathered examples of Dora’s speech to help the model capture her tone, rhythm, and delivery. This let us connect the AI’s teaching responses to the character: Dora could respond to what a child actually said, using dialogue that was never in the original episode.
We connected that voice model to our OpenAI reasoning pipeline. The reasoning model uses the scene, question, and child’s answer to decide what Dora should say; the voice model turns that response into speech. Bringing those pieces together made the cartoon’s questions into real conversations.
Why we chose the Tiger Data track
Every answer a child gives creates a learning event: the question, the answer, whether they needed help, and the skill it practiced. Over time, these events become a record of how the child is progressing.
We store this data on Tiger Cloud in a TimescaleDB hypertable. Each learning event is written in the same transaction as the child's answer, so the dashboard always stays consistent. The parent dashboard uses time_bucket() to group events by day and week, which lets parents see, for example, whether counting is getting easier or shapes still need practice. Since Tiger Cloud is Postgres, our sessions, answers and skills live in the same database as the time-series data.
Challenges we ran into
Training Dora’s voice was one of our biggest technical challenges. Our early results generated speech, but it was off.
Aman worked on gathering more Dora voice data and refining the model to improve that resemblance. We had to think carefully about both the voice quality and how quickly we could generate each response. Even a convincing voice would break the flow of the cartoon if the child had to wait too long to hear it.
That made latency a challenge across the entire system: understanding the child’s answer, generating a useful explanation, and producing Dora’s audio all had to happen within the same conversational pause.
The reasoning pipeline had a similar tradeoff. A response needed to arrive quickly enough to keep the conversation moving, but it also needed to make sense, address the child’s answer, and give useful guidance. We worked on supplying the right context and constraining the responses so Dora could stay concise without losing the explanation.
Lexi’s challenge was organizing all that information into a clear parent experience. We had individual answers, learning moments, skills, and curriculum categories, and we needed to agree on an information hierarchy that made them easy to understand. Deciding what parents should see first was as important as deciding what we could store.
What we learned
We learned that making an AI character respond is only one part of making the interaction work. The voice, timing, explanation, and return to the episode all have to fit together. A good answer can still feel awkward if it arrives too late or sounds unlike the character.
We also learned that collecting learning data and communicating it are different problems. Parents need a clear picture of what happened and where their child might need more practice. A correct answer after help is useful evidence, but it does not tell the whole story of what a child understands.
Our original idea was to make Dora listen. Building it pushed us to think about the whole experience: how kids participate, how feedback helps them, and how parents stay involved.
Built With
- cloning
- convex
- css
- drizzle
- elevenlabs
- natural-language-processing
- next.js
- openai
- postgresql
- react
- recognition
- speech
- tailwind
- text-to-speech
- tiger-data
- timescaledb
- typescript
- voice