Rhubarb

A tutor that listens through your Bee, turns your real conversations into lessons, and hears you get better when you use what you learned.

Comment

Inspiration

Most of what you could learn from your day is gone by dinner: a word you didn't know, a question you couldn't answer, a clerk you couldn't follow. On a trip to Japan I wore a Bee and realised it hears all of it. What if a tutor started from your real day, and counted progress by whether you use what you learned?

What it does

Rhubarb is a tutor that listens through your Bee and builds learning plans from your own day, for a language, a work skill, or anything you're curious about.

  • Listens: reads the Bee's live transcript. Text only, never audio. Private moments are dropped before any AI sees them.
  • Checks with you: every evening it shows what it thinks happened. You mark each finding right or not quite, and fix anything Bee misheard. Nothing is built on a guess.
  • Teaches: suggests plans from what you confirmed. Say yes and how much you already know, and it writes a short lesson around your own words, fact-checked by a second AI.
  • Practises through the Bee: your phone buzzes with a question, and you answer out loud into your Bee.
  • Hears you get better: later conversations are checked for the skill used for real. That's the progress, not streaks.

Also: ask out loud to get a lesson on anything, quick "guess first" answers, places (a landmark, never your address), reading help per plan, a day-by-day look back, and forget anything.

How we built it

  • Bee: bee stream --json for the live transcript and voice notes, the Bee API to recover missed lines, todos with alarms to reach the phone, and conversation locations for places.
  • Claude on Amazon Bedrock: Opus writes, fact-checks and grades; Sonnet sorts what was heard. Anthropic's API is an automatic fallback.
  • Stack: TypeScript, Node, SQLite, React, three.js. Runs on Railway around the clock; the judges' demo is on Vercel. Tavily for sources, 68 tests, a daily cost cap.

Challenges we ran into

  • Bee deletes short conversations minutes after they end, so we capture everything live.
  • The live stream drops about every 90 minutes, and nightly transcripts use different ids, so we built a supervisor and matching by time and text.
  • Bee can mishear ("Osaka" became "Old Southfield") and doesn't always know who's speaking. That's why the evening check exists.
  • A new AWS account couldn't call any model until AWS Support enabled them.

Accomplishments that we're proud of

  • It works on real life: across six days of wearing the Bee, 6,978 lines heard, 20 plans suggested and 7 lessons written and fact-checked, in Japanese and English. ## What we learned

A first reading of someone's day is a guess. Rhubarb got much better once it asked before assuming.

What's next for Rhubarb

  • Use Bee's voice recognition to tell your lines from everyone else's.
  • A phone-only mode, no wearable needed.
  • Bring it to adult English learners, with real-world use as the measure of progress.

Built With

Share this project:

Updates

Submission history