Memory Steward

A self-hosted MCP server that gives Alexa+ a long-term memory you own: a plain file in your Google Drive, with no silent writes, history kept, and exports you control.

Comment

Inspiration

Every assistant now wants to remember you, and each one keeps that memory to itself: on its servers, in a format you can't read, changed whenever the model decides. I had been keeping my own context as plain documents in Google Drive so several assistants could read it. Memory Steward turns that habit into a proper memory layer for Alexa+: the assistant manages the memory, but the user owns it.

Voice makes the problem sharper. Speech recognition mishears, models over-infer, and a fact said once ("the contractor starts on the 20th") quietly goes stale. A voice memory needs stricter rules than a chat memory, not looser ones.

What it does

Memory Steward is a self-hosted MCP server that Alexa+ links to with OAuth. The memory itself is one human-readable Markdown file in the user's Google Drive. Four rules are built into the tools, so they hold no matter which model calls them:

  1. No silent writes. propose_memory returns a sentence to read back; nothing is saved until confirm_memory carries the user's yes.
  2. Facts change, history stays. When a new fact looks like an update, the server asks one question. One yes saves the new fact and keeps the old one as history instead of deleting it.
  3. Temporary facts get re-checked. Facts can carry a review date. When it passes, Alexa asks "Is that still true?" in a question the server writes, and waits for the answer.
  4. It's really yours. Edit the file by hand in any text editor and Alexa uses your edit seconds later. Export exactly the part you choose ("share my food preferences") as a clean Markdown file any person or AI can read. The model never sees the export; it goes straight to the screen.

Ask "What do you know about me?" and it gives an honest overview instead of guessing. The server writes every sentence meant to be spoken in the second person, so any model, including the one inside real Alexa+, can read it out as given.

How I built it

The Alexa+ MCP Toolkit and devices are partner-gated, so the judged experience is a simulated Alexa+ web app that is a real MCP client of the real server. Everything the server does is built to the published Alexa+ requirements, so it is ready for a real listing once access opens.

  1. The simulated Alexa+ takes speech through the browser, then runs an Amazon Bedrock Converse tool-use loop on Amazon Nova Pro. It links the account through OAuth exactly as Alexa account linking does, and loads the memory rules from an Agent Skill (SKILL.md) with progressive disclosure.
  2. The MCP server (Python MCP SDK 2.x, Streamable HTTP) exposes 8 tools: recall, memory_overview, propose_memory, supersede_memory, confirm_memory, list_due_reviews, resolve_review and export_memory, plus the memory file as an MCP resource. Reads come from an in-memory cache; writes reach Google Drive through a write-behind thread, so no tool call ever waits on Drive.
  3. The memory is memories.md in the user's Drive folder, accessed by a service account that can see only that folder. Hand edits are picked up by a version-aware refresh, with a merge when both sides changed.

Alexa+ readiness (each item mapped to code and a test in docs/ALEXA-READINESS.md):

  • Streamable HTTP, MCP spec 2025-11-25 or later: pass
  • Tool latency under 500 ms: pass (1–2 ms locally, 3–12 ms inside AWS, about 75–105 ms from a laptop over the internet)
  • OAuth 2.1 authorization code + PKCE (S256), resource parameter, rotating refresh tokens: pass
  • 401 without WWW-Authenticate, RFC 9728 protected resource metadata, authorization server metadata: pass
  • No dynamic registration or OIDC, one pre-registered client, scopes enforced per tool: pass
  • Listing artifacts (addon.json, icons, carousel image, privacy and terms pages): drafted
  • End-to-end with real Alexa+: not possible yet (partner-gated)

AWS usage:

  • Amazon Bedrock + Amazon Nova Pro power the simulated Alexa's reasoning and tool use; Nova Lite plays the "different assistant" that reads an export.
  • Amazon ECS Express Mode on Fargate (ARM64) hosts both public services behind one HTTPS load balancer, with an IAM task role scoped to Nova Pro only (no static keys).
  • Amazon ECR, SSM Parameter Store, CloudWatch Logs and AWS Budgets handle images, secrets, logs and cost control.
  • Amazon Polly generated the narration and voices in the demo video.

Quality gates: 158 automated tests, plus a Bedrock evaluation of the golden conversations on Nova Pro (24/24 locally, 9/9 on the public deployment for the final build).

Challenges I ran into

  • No access to real Alexa+. The MCP Toolkit, CLI and devices are partner-gated, so I built a simulator that behaves like Alexa (account linking, tool use, spoken replies) and held the server to the published checklist instead.
  • Instructions are not guarantees. Nova Pro sometimes read an exported fact aloud despite a rule against it, answered its own "is this still true?" question, or wrapped replies in markup tags. Each fix moved the rule out of the prompt and into the design: exports travel in the tool result's _meta so the model never receives them, the server writes every spoken question itself, and replies are cleaned of any markup before speaking.
  • The MCP SDK's built-in auth conflicted with Alexa's rules (it always sends WWW-Authenticate, treats resource as optional, assumes dynamic registration), so I wrote a small standard-library OAuth 2.1 server instead.
  • Google Drive realities: service accounts cannot own files in a personal Drive and there is no conditional upload, so the store never creates files, uses version-aware refresh and merges conflicting edits.

Accomplishments I'm proud of

  • Every tool call runs in milliseconds against Alexa's 500 ms budget, even while Drive uploads run in the background.
  • One "yes" handles a correction, with full history kept: safer and shorter than two confirmations.
  • The privacy guarantee for exports is structural, not a prompt: the model cannot leak what it never sees.
  • A friction log, a known-issues table, reproducible proofs for every claim, and evaluations that check the tool calls on every turn, not just the final file.

What I learned

For voice, the safest place for a rule is the tool contract. If the server writes the sentence to speak and decides what the model can see, any model behaves well, including ones you don't control, like the model inside Alexa+.

What was built during the hackathon

The idea came first: before the hackathon I kept personal context as plain documents in a Google Drive folder. No code existed. Everything in the repository was written during the submission window (first commit Oct 14, 2026). The demo uses a fictional user, not my own notes.

What's next for Memory Steward

  • Live multi-assistant sharing: several MCP clients linked to the same memory with per-client attribution (today, exports are the cross-platform path).
  • Household memory with per-person scopes on a shared Echo.
  • A real Alexa+ listing once MCP Toolkit access opens.

Built With

Share this project:

Updates

Submission history