SayBetter

An English speaking coach built from what you actually said today: Bee on your Apple Watch hears it, SayBetter fixes it, and your wrist drills it tomorrow.

Comment

Inspiration

I've studied English with Duolingo and Speak, the Korean speaking app. They grade the sentences they give me, and much of their feedback is about pronouncing a scripted line. But when I talk for real — on a call, at work — what trips me up is different: under pressure I say "Yesterday I go…", "I am agree", "explain me". No app hears those moments. More and more learners now practise with AI tutors instead of apps, and I wanted that for the English I actually speak. Bee already listens and transcribes my day on my Apple Watch, so the missing piece was a coach that learns from that — and that can tell whether a fix in practice carries over to real conversation.

What it does

SayBetter turns the English you really spoke into a daily, personal lesson — and checks whether it sticks in real life.

  1. Listen. saybetter watch follows the live Bee stream from the Apple Watch and keeps only English (including English inside Korean code-switched speech). ASR often splits one sentence into fragments; SayBetter reads neighbours together.
  2. Coach. Within seconds an LLM returns, for each sentence worth fixing: a minimal fix, what a native speaker would actually say, a short explanation in Korean (my first language), a mistake pattern and a severity — with a macOS notification. It deliberately skips ASR noise and caps corrections so it stays motivating.
  3. Practise. A calm web notebook shows today's sentences with a word-level diff, opens with "Today's one thing", and has a Practice mode that follows corrective-feedback research: rule first, then a cloze hint, then the answer; you say it (Web Speech API) or type it. Passed sentences go into Leitner spaced repetition (1, 3, 7, 14 days).
  4. Say it out loud. In Practice you say the corrected sentence; SayBetter records it, runs whisper.cpp locally with word-level probabilities, aligns it with the sentence you meant, and marks the words a listener would miss in red (heard as a different word) or amber (unclear), with Korean tips for the sounds Korean speakers find hard (th, r/l, v/b, final -s/-ed). Try again up to three times — the Duolingo/Busuu principle, but on your own sentences.
  5. Drill on the wrist. saybetter review turns the top corrections into Bee todos with alarms tomorrow at 09:00, 13:00 and 19:00 — "Say it: Last week we met the new team…". When the alarm time comes, Bee sends it as a reminder — rewritten in its own words ("Hey Jinsu, sorry to jump in — just a quick nudge for your English practice. Time to say it out loud: 'That's a good point.'") — and with the iPhone locked it buzzes on my Apple Watch (tested on my SE 3 with a test alarm; while I am using the iPhone it shows there instead). The SayBetter Drill app on my Apple Watch SE 3 reads those drills live from Bee todos; you say the sentence into the watch, the Mac scores it with whisper.cpp, and on a pass the Bee todo is completed and the Bee fact records the practice ("practised out loud on my Apple Watch 1× (latest pronunciation score 93)"). Reopen a drill later and it still shows Done in Bee ✓ · 93% — Bee holds both what to practise and whether you did. The watch app can also score on the watch itself with WhisperKit (openai_whisper-tiny.en): its Settings switch between Auto (on-watch, falling back to the Mac), On watch and Mac server. On-watch scoring ran in the watchOS simulator, but the tiny model was much less accurate (67% on a sentence the Mac scored 96%), so the demo on the real watch uses Mac server.
  6. Remember. A pattern you slip on twice becomes a Bee fact, so your personal AI knows what you're working on.
  7. Prove it. The next time you say that pattern correctly in a real conversation, SayBetter detects a real-life win and updates the Bee fact: "Slipped 2×, got it right in real conversation 1×." That is the metric no other English app can measure.

Nothing in the demo is staged (the video is cut and sped up in places). You hear my real voice: a Mac mic recorded it while I spoke into my Apple Watch (Bee), while I practised in Chrome (real attempts: 100, then 90) and while I drilled on the watch, which my iPhone filmed (93%). The transcripts, todos and facts are from my Bee account. Only the narration uses my own cloned voice (fine-tuned Qwen3-TTS).

How we built it

  • Architecture: every Bee read and write goes through the Bee CLI on the Mac (as the Bee track asks: CLI, MCP or Agent Skills). The SwiftUI watch app runs on a real Apple Watch SE 3 and talks to the Mac over Wi-Fi; the Mac scores with whisper.cpp and writes to Bee.
  • Node.js, zero dependencies, 40 tests, GitHub Actions on Node 20/22/24.
  • Bee at runtime: bee stream --json (live utterances), bee conversations list/transcript (reviews), bee todos create --alarm-at (wrist drills), bee todos list / bee todos complete (the watch drill loop), bee facts create / bee facts update (memory + progress), plus a SayBetter Agent Skill so Claude Code or Codex can quiz you from Bee context over the Bee MCP server.
  • Coach: Claude through Claude Code headless (claude -p) by default; Gemini supported; a deterministic rule engine for offline tests. We benchmarked LanguageTool and Harper on real learner sentences — they caught 2/5 and 0/5 of the speaking mistakes the LLM caught 5/5 (tense with time words, articles, explain me) — so the LLM coaches and rules are the fallback.
  • Web UI: dependency-free SPA served by saybetter serve — Today, Practice, Patterns (slipped vs. got it right), Wrist drills with a watch mock; dark mode, mobile tab bar, accessible.
  • Habit loop: saybetter schedule --at 22:00 installs a launchd job for a nightly review.
  • Before building, we surveyed Speak, ELSA, Ringle, Loora, Duolingo Max, Langua, 말해보카 and open-source tutors. Ideas we adopted: two-step correction, few corrections, one takeaway a day, elicitation before recasts, spaced repetition of your own sentences. What none of them do: learn from your whole day of real speech and verify transfer to real conversations.

Challenges we ran into

  • Apple Watch capture pauses when the screen sleeps; the conversation stays CAPTURING but nothing arrives.
  • ASR "fixes" learner grammar ("I didn't said" → "I didn't say") and splits sentences; a fragment once looked like a correct sentence and produced a false real-life win, so we made the coach read neighbouring fragments together and only count wins for complete sentences with the trigger and the correct form.
  • No speaker labels on the stream, so SayBetter must not coach other people; we document that and coach only the user's own English sessions.
  • Concurrency between a nightly review and a manual one double-scheduled drills; we capped drills per day and recreate a Bee fact if it was deleted in the app.

Accomplishments that we're proud of

  • It runs on my real Apple Watch SE 3, not only in the simulator: drills come from Bee todos and a pass goes back to Bee.
  • A closed learning loop on real data: spoken into an Apple Watch → coached in Korean → drilled on the wrist → remembered by Bee → verified in a later real conversation.
  • It feels like a product: a quiet, readable notebook rather than another chatbot.

What we learned

Bee's CLI + facts + todos are an excellent substrate for personal learning: the todos come back as Bee reminders on the wrist, and facts let the learner's own AI carry their goals. The hardest part of wearable learning is not the model — it is trustworthy capture (pauses, ASR smoothing, fragments).

What's next for SayBetter

  • Open the watch drill straight from the Bee alarm (needs an alarm action/deep link in Bee), and make on-watch scoring accurate enough to drop the Mac (WhisperKit tiny.en is weaker than the Mac's large-v3-turbo)
  • Phoneme-level pronunciation scoring
  • Speaker-aware coaching once Bee exposes speakers on the stream
  • Alexa+ / Fire TV: "What's my one thing today?" and a morning drill on the TV
  • Other language pairs (Japanese/Spanish learners)

Built With

Share this project:

Updates

Submission history