Hearsay

Hearsay turns what you say into work: Bee on your Apple Watch hears "I need to fix the login bug after lunch" and files the issue.

Comment

Inspiration

I narrate my work all day: "after lunch I need to fix that redirect bug", "remind me to email the professor tomorrow". On calls, to friends, to myself. Almost none of it survives until I sit down at the keyboard. Bee already captures and transcribes what I say on my Apple Watch, so the missing piece was turning those sentences into the places where work actually happens.

What it does

Hearsay turns what you say into work.

  1. Listen. Hearsay follows the live Bee transcript stream (bee stream --json) from an Apple Watch running the Bee app, buffering utterances per conversation.
  2. Understand. After a pause in the conversation, it asks an LLM for commitments only: software work, actions you must take, or facts worth noting. Small talk is ignored. Times like "tomorrow at ten" become real timestamps. Korean and English can be mixed in one sentence; each item keeps the language it was spoken in.
  3. Act.
    • Software work → a GitHub issue with the exact spoken quote attached
    • Anything else you must do → a Bee todo (with an alarm when a time was mentioned)
    • Facts → a local note
    • Every item → a macOS notification and a line in ~/.hearsay/log.md
  4. Close the loop. A Claude Code skill reads the same Bee context through the Bee MCP server, shows you what you said today, and starts on the issue you pick — with the spoken context in hand.

Say it → it's filed → your coding agent picks it up.

How we built it

  • Node.js, zero dependencies. Every Bee call goes through the official Bee CLI so the project uses whatever auth bee login set up.
  • Bee at runtime: bee stream (live utterances), bee conversations list / transcript (replay of recent conversations), bee todos create (todos with alarms), and the Bee MCP server inside Claude Code (bee_search, bee_get_conversation_transcript, bee_list_todos, bee_complete_todo).
  • Intent extraction runs through Claude Code in headless mode (claude -p) by default, with Gemini and an offline keyword backend as alternatives.
  • A state file remembers handled utterance IDs and recent titles, so restarts and replays never file the same task twice.
  • hearsay replay re-processes recent conversations when no live stream is running; hearsay extract tests the extractor on one sentence.
  • 11 unit tests, MIT licensed.

Challenges we ran into

  • Getting the CLI approved. Developer Mode (tap Version five times) gives no feedback the first time, and the bee login link opened in iPhone Safari shows only a QR code. What finally worked: show the QR on the Mac and scan it with the iPhone camera, which opens the Bee app's approval screen.
  • An undocumented event schema. I captured raw stream events to learn the real shape: utterances carry a conversation_uuid, while summary events carry a numeric conversation_id, and stream utterances have no speaker. The parser is defensive and every raw event is logged for debugging.
  • Real speech is mostly not tasks. On real phone calls the extractor must return nothing almost all the time. The prompt is tuned for precision: only commitments the speaker made, no hypotheticals, no questions to others.

Accomplishments that we're proud of

  • End to end on real data: words spoken into an Apple Watch became a GitHub issue in the demo repository and a Bee todo with an alarm, with no typing.
  • About the demo video: the narration and the spoken sentence use my own voice cloned with a fine-tuned Qwen3-TTS model, played from the Mac speaker to the Apple Watch. The Bee transcript, the GitHub issue and the Bee todo shown are real and unedited.
  • It works in Korean and English, which is how I actually talk.

What we learned

Bee's CLI and MCP are unusually agent-friendly: every command has --json, and the MCP tools map one-to-one to CLI commands. Wearable context is most useful when it lands inside existing tools (issue tracker, todo list, coding agent) rather than in yet another app.

What's next for Hearsay

  • Daily stand-up summary from bee today --context
  • Per-person follow-ups after meetings
  • A confirmation step on the watch before filing an issue

Built With

Share this project:

Updates

Submission history