Katasaga

• 

  The home screen places your current story directly into the world, with progress, streak and daily learning integrated into the scene.

• 

• 

  Illustrated scenes combine narration, music, ambient sound and interactive dialogue to move the story forward.

• 

  Speaking practice happens inside the story. Learners speak with characters in real time while using resources to complete the scenario.

• 

  At key moments, the story hands the conversation to the learner, with real-time voice interaction that lets them shape what happens next.

• 

  The speaking page uses an explorer's desk as the navigation surface, with each object leading to a different type of practice.

• 

  Spaced repetition becomes part of the world, with your review waiting for you as a stack of cards on the explorer's desk.

• 

  After the conversation, Katasaga evaluates the interaction against the scenario criteria and explains what the learner accomplished.

• 

  Progress is presented as a physical notebook, with goals and stats integrated directly into the artwork.

• 

  Progress is presented as a physical notebook, with goals and stats integrated directly into the artwork.

• 

• 

  Each speaking session ends with a breakdown of what the learner learned, including vocabulary, grammar and areas to improve.

Inspiration

I noticed that even after years of classes or long streaks in apps, I still felt stuck when it came to actually speaking. Lots of vocabulary and grammar, but not enough chances to use it.

So where do you go if you want to practice speaking while actually feeling immersed in a language?

That’s why I started building Katasaga. An app where you get immersed in a different world through stories and, along the way, get real opportunities to start speaking confidently.

What it does

Katasaga is a language learning experience where you step into voiced stories, speak with characters to move forward, and keep what you learn through spaced repetition.

Instead of points or rewards, the focus is on building the confidence to actually speak.

Stories

Follow illustrated, voiced scenes with music and ambient sounds that match the setting. Translate any word and move through the story scene by scene.

Speaking Practice

At key moments, the story hands the conversation to you. Speak with characters in real time, with your response shaping what happens next.

Knowledge Gap Capture

Words and phrases you struggle with are automatically turned into flashcards and brought back through spaced repetition.

Immersion

Sound, motion, and small quirks like page-turn transitions and subtle callbacks make the world feel lived in, without turning it into a game.

Beyond the story, you can also practice through freeform conversations, scenarios, and CEFR evaluations!

How we built it

Core: React + TypeScript, with Supabase for data, accounts and storage, and Capacitor to share the same codebase between web and iOS.

Live conversations: LiveKit with a Python voice agent and custom pipeline, connecting speech recognition, conversation and character voice into a real-time loop. This also handles the automatic flashcard generation.

We separated the conversational actor from the background judge, so the character stays in role while the learner is evaluated without interrupting the conversation (Safety guardrails also run in parallel)

Design & motion: We built the interface around artwork, sound and expressive motion, using Motion, Anime.js and CSS for everything from small touch feedback to larger scene transitions.

Subscriptions & engagement: RevenueCat handles subscriptions, while OneSignal powers contextual notifications designed around Katasaga's story-first philosophy. See below for how OneSignal became part of that experience.

Challenges we ran into

From AI tutor to story-driven learning

Katasaga started as an AI tutor with automatic flashcards. I quickly realized that a blank conversation still asks the learner to motivate themselves and figure out what to talk about. That led me to build the experience around stories and characters, so speaking has a reason and a place within the story.

Getting the AI to behave

This was probably my biggest technical pain point. I spent a lot of time tweaking the actors, judges, prompts and guardrails, often fixing one problem only to find another.

The judge was initially far too strict, sometimes failing valid answers, while the summary agent would hallucinate why someone had failed. I eventually separated the conversational actor from the background judge, so evaluation could happen without breaking the conversation.

Accomplishments I'm most proud of

The biggest thing I'm proud of is making the embedded speaking scenarios feel like part of the story, rather than separate language exercises.

I'm also proud of the systems around them: automatic flashcard creation and spaced repetition, the way artwork becomes part of the UI, and the amount of sound, music and small interactions that make the experience feel alive.

Most importantly, I managed to turn the original idea of an AI tutor into something that feels like a complete language learning experience , I'm really happy about the end result!

What I learned

I learned that building with AI is less about getting the perfect prompt and more about building systems around the model. Actors, judges, guardrails and evaluation all need to work together, and small changes can completely change the experience.

I also learned how much iteration matters in design. A lot of Katasaga's interface came from showing it to people, seeing what felt natural, and changing it again.

What's next for Katasaga

I want to keep writing new stories and embedded speaking scenarios based on what users are interested in, whether that's romance, drama, or a spy thriller.

I'm also working on better pronunciation and understanding of non-native accents, as well as oral exam simulations for more structured speaking practice.

Now that most of the core technology is in place, I finally get to unleash my creative side! I hope to collaborate more with artists and the Katasaga community to build out this world of stories :)

Built With

• ai
• anime.js
• audio
• capacitor
• conversational-ai
• css
• interactive-stories
• ios
• language-learning
• livekit
• mobile-app
• motion
• natural-language-processing
• onesignal
• python
• react
• real-time-voice
• revenuecat
• spaced-repetition
• speech-recognition
• storytelling
• supabase
• text-to-speech
• typescript
• voice-ai