Storykeeper

A family's living memory, told to Alexa: elders talk, families get the recipes, health history, papers and stories they'll one day need.

Comment

Inspiration

When a grandparent passes away, a family loses more than stories. Nobody knows where the house registry is, which bank has the locker, which medicines Nanaji took, what he was allergic to, or how Nani made the pickle. The knowledge was never written down, because elders don't type into apps. But they will talk, and Alexa is already in their living room.

What it does

Storykeeper is an Alexa+ integration, built as a self-hosted MCP server, that interviews elders and turns what they say into things a family actually uses.

  • Interviews that know what's missing (Memory Gap Engine). Every possible gap is scored by importance × uncertainty × freshness. Examples: no allergy on record, nobody knows where the property papers are, nothing yet from the 1980s, metformin was mentioned 5 months ago and may have changed. Alexa asks about the most valuable gap first, the family dashboard shows the ranking and the reason, and the elder can skip anything. A skipped question sinks instead of nagging.
  • Five shelves. Stories · Kitchen · Health · Where things are · Wisdom & people. Amazon Nova on Bedrock files each answer, dates it from age clues ("when I was fourteen"), and extracts structured details: ingredients and steps, condition and onset age and medicines, item and location and who to call.
  • Speak the way they speak. Punjabi-English, Hindi-English, Gurmukhi or Devanagari. The original words are kept exactly and a faithful English translation is stored beside them. Cultural terms stay as said (saag, ghotna, baraat). Recipe cues like "until the khushboo comes" are kept, and quantities are never invented.
  • Every fact has a source. Each memory shows who told it, when, and how. Each extracted fact is tagged in their words or inferred.
  • Two versions, never silently merged. Nanaji remembers the wedding as 1970 and Nani remembers 1971. Storykeeper keeps both, shows them side by side, and says both when a family member asks. Nothing is resolved until the family settles it.
  • Relatives suggest, the elder approves. A granddaughter can propose "it was 1970, the wedding card says so". Nothing changes until the elder approves. Their original words are never edited, and every decision is kept as history.
  • Ask the archive. "Alexa, ask Storykeeper where Nanaji kept the LIC policy." → "In the blue folder under the bed. His agent is Mr. Bhatia from Ghumar Mandi." Answers come only from the elder's own recorded words, and every answer cites its source memories.
  • Print for the family. Doctor's sheet (health history, onset ages, medicines, allergies), "If something happens" sheet (where the papers, keys and policies are, who to call), recipe book, and memoir chapter.
  • Family graph. People, places and historical events linked by the memories that mention them. Historical events are shown as context, never as something the elder said.
  • Old photos and their real voice. Show the elder an old photo during an interview and it's kept with their answer. Record their actual voice alongside a memory, so grandchildren hear them, not a synthetic narrator.
  • Safe by design. Account numbers, card numbers, Aadhaar/PAN, PINs, OTPs, passwords and phone numbers are stripped before anything is stored or sent to the model, including in Punjabi or Hindi sentences ("mera pin 7790 hai"). The vault records where the locker key is, never the PIN.

How we built it

  • MCP server: Python with the official MCP Python SDK (MCPServer, Streamable HTTP at /mcp, protocol 2025-11-25 supported). It exposes 10 tools: next_interview_question, save_memory, ask_family_archive, get_memory_gaps, get_life_timeline, get_family_handbook, list_family, get_family_graph, suggest_correction, review_suggestion. Tool descriptions and server instructions tell Alexa+ how to run the interview loop and what to speak back.
  • One process, two front doors: the same server hosts the MCP endpoint for Alexa+ and a JSON API for the family web app, so both share the same tools and data.
  • AI: Amazon Nova 2 Lite on Amazon Bedrock through the boto3 Converse API. Responses are requested as JSON against a schema, then validated and coerced. If Bedrock is unreachable, every call falls back to deterministic heuristics so the experience never dead-ends.
  • Memory Gap Engine: deterministic and explainable scoring, which works even without a model. The model is given the ranked gaps and phrases the question naturally.
  • Storage: a single SQLite file the family controls, with regex redaction before storage, keyword retrieval, provenance and conflict detection.
  • Web app: vanilla HTML/CSS/JS with an original animated illustration, life ribbon timeline, page-turn book, family graph (SVG), browser speech synthesis for the narrator, and MediaRecorder for the elder's real voice.
  • Tests: 18 offline unit tests covering redaction (and proof a secret never reaches the model or the database), provenance, conflict detection, gap ranking, upload checks, the approval workflow and the graph.

Challenges we ran into

  • Our brand-new AWS account (AWS India / AISPL) passed customer verification, but Amazon Bedrock still refused model invocation for the whole hackathon window: "Your account is currently being verified" in some regions, "Operation not allowed" in others. We designed Storykeeper so every Bedrock call has an offline fallback, and the demo runs in that mode. The Bedrock integration is complete and switches on automatically when the account is cleared.
  • There is no way yet to point a real Alexa+ device or simulator at a self-hosted MCP server, so we built a simulated Alexa+ panel that calls the exact same tools.
  • Handling code-switched speech honestly: keeping the original, translating faithfully, and never inventing quantities in recipes.
  • Telling families what the AI inferred versus what the elder actually said.

Accomplishments that we're proud of

  • An interviewer that knows what the family has not recorded yet, and why it matters.
  • Trust features (sources, "in their words" vs "inferred", two versions, elder approval) that respect that family memory is personal and not always consistent.
  • Privacy that is tested, not just promised.

What we learned

MCP makes it straightforward to give Alexa+ a rich, multi-step workflow. The hard part of a memory product isn't storage; it's trust: provenance, uncertainty, consent and never putting words in an elder's mouth.

What's next

  • Deploy the MCP server publicly and connect it to Alexa+ as soon as self-hosted MCP access is available.
  • Enable Amazon Nova once our account is cleared, and add Amazon Polly neural voices for Alexa-quality narration in the web app.
  • Per-relative permissions (a grandchild sees the recipe book but not the health or vault shelves).
  • Encrypted backups and export to PDF/JSON for long-term preservation.

Built With

Share this project:

Updates

Submission history