Sala, an Alexa+ MCP Client

A living-room friend who chats you into a new language — and turns your own sentences into your next lesson.

Comment

What this is

Sala is a language companion for the living room, built as a real MCP server for Alexa+. You chat with it in whatever language you're learning and it behaves like a friend, not a tutor: it remembers what you did today and what you said last week, keeps everyday conversation going with that personal context, and changes language when you ask out loud, mid-session. When you end a session you get a grammar and vocabulary review built from sentences you actually said, quoted verbatim.

What inspired us

Alan's framing was "a living room friend" for language practice. Two findings set the direction. A sweep of the Alexa+ language category turned up almost nothing. And the hackathon rules themselves use "maintains state across sessions" as their example of the creative bar, which is exactly what language retention needs. Memory became the product, not a feature bolted onto a chatbot.

How we built it

  • The server is the real integration surface. Alexa+ acts as the MCP client: the device streams audio to Amazon's cloud, ASR happens there, and a third-party add-on receives text. So a text MCP server is not a workaround, it is exactly what a production add-on looks like. We speak MCP spec revision 2025-11-25 over Streamable HTTP, the version the track sets as its minimum and the version Alexa+ for Builders supports, with three-way alignment to our pinned SDK (@modelcontextprotocol/[email protected]).
  • Six tools, all with typed zod input AND output schemas so clients get structuredContent: chat, log_activity, recall, end_session, set_language, list_sessions.
  • Memory on SQLite + FTS5, hosted anywhere. There is no third-party Alexa+ memory API, so portability is the point.
  • The web sim is a genuine MCP client, official SDK imported and called at runtime, labeled in-UI as a simulated Alexa+ experience. Its voice layer stands in for the Amazon-side stack: browser speech recognition for device ASR, browser or local Piper neural TTS for the speaker.
  • Five swappable LLM backends behind one interface: a zero-secret echo mode (the demo runs end-to-end with no API key and no spend), a proxy/Anthropic-compatible path, Amazon Bedrock wired as first-class code, and bring-your-own-key OpenAI-compatible and Gemini lanes added the night before the deadline so anyone can run it on their own key.

What we learned

  • Memory in a voice product is a trust problem, not a storage problem. Sala can only follow up on events found in YOUR turns, needs an event noun you actually named, and vetoes anything that sounds already finished. Missing a question beats inventing one. A partial unique index makes "asked twice about the same interview" a schema impossibility. Every review example is validated in code for verbatim containment before it can appear, and the "remembered" badge in the sim lights only when the server actually retrieved something.
  • Spoken control beats toggles: "let's speak French" mid-conversation moves the persona, the review at the end, and the device voice, without restarting the session. Language is now a property of the stored session, which fixed a real bug where every turn silently reset an ongoing French session.
  • A feature is not done until it is driven in the surface that gets filmed. Our green test suites missed a bug where a language switch asked in a session's first utterance was dropped, and a scroll behavior that clipped the voice orb out of frame. The browser pass caught both.

Challenges we faced

  • Chromium on Linux reports zero speech-synthesis voices, which made the machine look voiceless. Fallback: local Piper neural TTS (en/fr/de), with a /tts?lang= honesty fix so a language with no model says so instead of faking it.
  • Half-duplex voice turn-taking: the browser's silence detector ended turns mid-sentence. Capture is continuous now and only pressing the mic commits a turn.
  • end_session is a real LLM pipeline (roughly 30s per close with a live model), visible in the demo as a cut-around, stated here rather than hidden.
  • Staying honest under deadline pressure: the Bedrock integration is complete and documented, but no live Nova reply has been observed on our account shape, so we do not claim the AWS Builder mini-challenge. The BYOK lanes ship with 28 offline tests and an explicit "first live evidence is your key" note. The 100% offline demo path (echo mode) exists precisely so nothing in the video depends on a credential we cannot hand out.
  • 328/328 offline tests, re-verified inside a network namespace (loopback up, egress blocked), and the cross-session beats (the follow-up question, the recurrence count) were earned by real closed sessions in the demo database, not seeded rows.

Try it

git clone, npm install, npm run demo (echo mode, no keys), open the sim, talk to it. Full cold-run guide in the README, written for a teammate on a fresh machine.

Built With

Share this project:

Updates

Submission history