evoca

A voice-first family archive for Alexa+. Elders just talk; the whole family asks, records its own memories and reacts, while two AI agents build everyone's life chapters, privately.

Comment

Inspiration

Every family loses its elders' stories. A story told on a phone call stays with the one person who heard it, and the questions we always meant to ask are left for "next time". Existing memory products ask the elder to use an app, type answers or fill in a questionnaire, and they end as a one-way book on a shelf.

We wanted the opposite: the elder only has to talk, at the kitchen table, to a voice she already trusts, and the whole family, wherever they live, takes part. Alexa+ talking to third-party services through MCP made that possible: the device carries the voice, and everything else can live behind our own server.

What it does

Evoca is a voice-first family archive: an Alexa+ companion for the elder and a web panel for everyone else.

  • The elder just talks. At the family's chosen chat time the device lights up: "Elif has a question for you." The companion brings up the question at a natural moment, asks real follow-up questions, plays earlier stories on request ("play me what I told about the mulberry tree"), and tells her how the family reacted.
  • The whole family takes part. Any member can ask a question and follow its journey (asked → waiting for her morning chat → answered), record their own memory, react, and add an old photo. When the answer arrives, the person who asked gets a notification and hears just the part of her story that answers them, in her own voice.
  • Life chapters for everyone. Stories are sorted into life periods, not only for the elder: a memory a grandchild records about her grandmother and herself appears on both their chapters.
  • She doesn't have to be asked. If she tells the answer to a waiting question on her own, Evoca recognises it and sends it to the person who asked.
  • Privacy that the code keeps, not the model. "Keep this between us" marks exactly that part private; private text never reaches any model again. Sensitive topics set by the family are never brought up, and a question about one is not sent. A topic she refuses herself is never raised again. New stories wait 24 hours, so the family can hide one before it is shared.
  • Any family can start. A setup wizard creates the family, everyone joins with a personal invite link, and a "Getting started" card walks them through the first steps.

It is a loop, not a one-way book: question → conversation → her voice → reaction → the next conversation.

How we built it

  • A self-hosted MCP server (MCP Python SDK v2, Streamable HTTP, protocol 2026-07-28) holds all of Evoca's business logic behind 18 tools. The caller's identity comes from connection headers set by our backend, never from tool arguments, and every tool filters by access before anything reaches a model.
  • Two agents, both MCP clients, on Amazon Bedrock (Nova 2 Lite, Converse API with tool use):
    • the companion talks with the elder in real time;
    • the archivist runs after each conversation: it saves stories with their life period, matches answers to family questions, merges a memory the model split in two, links related stories and finds gaps to ask about later.
  • Every risky model decision gets a second, temperature-0 check: is this really a privacy promise, does this passage answer the question, is this story really a refusal. When unsure, the safe side wins. The opening's wording ("Your granddaughter Elif has a question for you") is composed in code and checked after the reply.
  • Voice pipeline: faster-whisper with word timestamps and family-name hints → the companion → Amazon Polly. Word-timed transcripts let us cut her recording to exactly the part that answers a question. A voice turn takes about 6 seconds end to end.
  • Web client: React + Vite. A demo layer (a person picker, a demo clock, a guided tour and sample answers in her voice) lets anyone run the whole loop in about three minutes, without a microphone.
  • Kiro was our IDE, and Kiro's agent built the foundation: the data model, access control, the first MCP tools, the chat agent and the voice pipeline. Our steering rules (.kiro/steering/) are shared by every coding agent we used, so the rules stayed the same from the first line to the last.
  • Quality evidence: 699 unit and API tests; a test that captures every request sent to the LLM and asserts no private text is in it; an end-to-end script that runs the whole demo on real AWS (9/9 steps); and agent evals where a model plays the elder across 17 scenarios, checked by code rules and a judge model.

Challenges we ran into

  • Our AWS project is locked to one Region. Polly's neural voices and Bedrock's image models were out of reach, and upgrading to the Paid plan did not lift the lock. We kept Polly's standard engine and documented every step in our friction log.
  • Models are confident about names. Speech-to-text heard a friend's name, "Ayten", as "died", and the story summary followed the transcript faithfully. We traced it to the transcript, not the archivist, and now say so in our known limits.
  • AI coding assistants kept writing MCP SDK v1 code (FastMCP, inputSchema), which failed only at runtime. A short "MCP SDK v2" section in our Kiro steering file stopped it.
  • Hidden tool errors: MCP SDK v2 reports any exception other than ToolError as a generic message, so the agent couldn't correct its own calls until we wrapped every expected error.
  • Our own eval had a bug. A stray backspace character where \b belonged meant two playlist checks could never fail. We fixed it, re-ran the scenario and say in the README that earlier results used a broken check.
  • Kiro's free credits ran out mid-task after four intensive days, so we finished the later features with another coding agent under the same steering rules.

Accomplishments that we're proud of

  • Privacy promises enforced in code, with tests that inspect every request sent to the model.
  • The family hears her real voice, cut to the exact words that answer their question.
  • A complete loop that really runs: a question from a phone, a conversation by voice, a story written up by a second agent, an answer in her voice, a reaction, and the next morning she hears it.
  • Honest evidence: every number in our README comes from a run, including the ones that don't flatter us.

What we learned

  • The model names the intent; the code does the precise work. Letting the model pick "keep this private" and the code compute the exact segments was far more reliable than asking the model for timestamps.
  • A second, tiny check beats a longer prompt. One-word temperature-0 checks caught mistakes that prompt rules kept missing.
  • Evals need evals. A broken check looks exactly like a passing one.
    • Steering files are the right place for project rules: one short file stopped a mistake that had come back three times in one day.

What's next for Evoca

  • A real Alexa+ device once third-party tools open: the system stays, the simulated screen goes.
  • Gathering mode: at the holiday table, everyone's phone becomes a microphone, and two versions of the same story are kept side by side.
  • More languages: the data model already stores each person's language, so a grandmother could tell her stories in her own.
    • A memorial archive: when an elder passes away, the family keeps her voice, and what she kept private stays private forever.

Built With

Share this project:

Updates

Submission history