Hearth

Inspiration

Families who live away from an aging parent worry about the same things every day: did Mom take her pills, when is her next appointment, is she okay? The usual answers are constant phone calls, or apps that older parents don't use. Voice is the one interface that works for both sides, so I built the care layer an Alexa+ assistant could talk to.

What it does

Hearth is a self-hosted MCP server for one household with two roles:

• Mom (the parent) speaks naturally: "I took my pills", "Any messages for me?" She has 8 tools and controls what her family can see. She can pause sharing of her medications or appointments at any time.
• Arnav (the caregiver) asks "How's Mom doing today?" and gets a short spoken summary. He can manage medications and appointments, send check-ins and acknowledge alerts, using 11 tools.
• Gentle escalation: a missed dose first sends Mom a nudge. If it is still missed 30 minutes later, the caregiver gets an alert.
• Nothing important changes without consent: every risky write (adding a medicine, booking or cancelling an appointment) is read back through MCP elicitation, for example "Add Physio Thursday at 10 AM? Say yes to save.", and only commits after a spoken yes. The server enforces this, not the LLM.

How we built it

• hearth-mcp: TypeScript on Node 22, @modelcontextprotocol/sdk 1.32.1 (MCP spec 2025-11-25) over Streamable HTTP, and SQLite through node:sqlite.
  • Each session gets only its role's tools, so an out-of-role call is simply "tool not found".
  • Natural dates are parsed with chrono-node and Luxon in the household's time zone.
  • Every spoken reply is 25 words or fewer and never contains an ID.
• voice-sim (MIT, generic): a browser push-to-talk harness that drives any MCP server through an LLM (Gemini or Groq).
  • Speech is transcribed with Whisper.
  • It includes a headless eval CLI that scores tool routing, confirmation compliance and brevity against a golden set.
• Delivery: one-command Docker Compose, plus CI on every push (typecheck, lint, tests, mock eval, metric thresholds, secret scan).

Challenges we ran into

• Free-tier LLMs moved under me: a pinned Gemini model started returning 404, a Groq model was retired mid-build, and Groq rejected some tool calls with HTTP 400. voice-sim now falls back between providers automatically.
• Speech recognition: Chrome's recogniser misheard short words in my accent ("log" became "locks"), so I moved to Whisper. I then found that Whisper invents words from silence, so the browser only sends clips that contain real speech.
• All 18 issues are documented in a friction log in the repo.

Accomplishments that we're proud of

All numbers are measured and listed in the README:

• Tool routing: 24/24 on the golden set. Confirmation compliance: 24/24. Replies of 25 words or fewer: 24/24.
• Tool latency p95: 51 ms. Median end of speech to spoken reply: 2.56 s.
• Fresh clone to a working MCP Inspector connection: under 5 minutes.
• voice-sim is MIT-licensed and has nothing Hearth-specific: point it at any Streamable HTTP MCP server to talk to it by voice and score it with a golden eval.

What we learned

Trust has to live in the server. An LLM will sometimes skip a confirmation or guess a date, so Hearth re-validates every plan and asks the human through elicitation before anything is committed.

What's next for Hearth

Connecting Hearth to Alexa+ as a production skill, real notifications to family members, and a confirm-token fallback for clients that don't support elicitation.

Code: https://github.com/Arnavshah22/hearth

Built With

• docker
• gemini
• github-action
• groq
• mcp
• node.js
• sqlite
• typescript
• web-speech-api
• whisper