EchoDock

Alexa sits silent while you use your computer. EchoDock (an MCP server) transforms it into a true desk secretary: briefing your day, launching workspaces, surfacing files & emails, and guiding focus.

Comment

Inspiration

Alexa runs my morning: alarm off, weather, oat milk on the list, music on. Then I sit down at my computer, and she goes silent for the next eight hours. Yet that's where my day really happens, and it's full of small chores: reopening the same apps and windows every time I switch tasks, digging through folders and email, and googling where an app hides a feature. Each one takes only a minute, but together they eat hours and break my focus.

I didn't want another assistant to learn. I wanted the one I already talk to. So I built EchoDock, an Alexa+ MCP server that turns Alexa into a voice-first desk assistant. One sentence sets up a workspace, files land where they belong, and hidden app features get pointed at on screen. One Alexa for my whole day, from the morning alarm to "good night."

At a glance: 30 MCP tools · 0.89 s median response · ~$0.0018 per request (about $3/month) · 130/130 on our routing eval · cited answers from your own files · on-screen guidance inside any Mac app with menus

What it does

One sentence replaces minutes of clicking.

  • Plans your day: "Good morning" shows today's calendar, the email that matters and your to-dos. At night, "What did I get done today?" sums it up.
  • Sets up your work: "Open my proposal workspace" brings back every app, document, tab and window position at once.
  • Finds and files things: "Open the PDF I downloaded this morning." "Tidy up my downloads." In a Save window, "Where should this go?" picks the folder. Every move can be undone.
  • Answers from your own stuff: "What did Dr. Morgan say about the proposal?" It searches your files, email, notes and calendar, answers in one sentence and names the source. If the answer isn't there, it says so instead of guessing.
  • Guides you inside any app: "I deleted a paragraph yesterday, can I get it back?" EchoDock reads the app's real menus, finds File › Revert To › Browse All Versions…, draws an arrow on screen, and clicks it when you say "do it." If it's not in a menu, it checks the window's buttons, then a screenshot (vision on Bedrock).
  • Gives you a shortcut pad for every app: the docked phone shows the best buttons for the app in front or you can customize it yourself.
  • Polishes writing right where you type: select text in any app and say "make this more professional." The edits appear marked, and "use it" drops them in. No more copy-pasting into a chatbot tab and back.
  • Manages your email: "Anything important?" sorts what matters. "Clean up my inbox" proposes a checklist and changes nothing until you say yes.
  • Guards your focus: a timer that pauses when you step away. The camera video never leaves the phone.

Every answer arrives three ways: you hear it, an arrow points at it, and a card appears on the Mac and the phone.

Why not Siri or Copilot?

Siri / Copilot EchoDock
"Where is X in this app?" Opens the app; Copilot Vision needs you to share the window first Reads real menus (exact for your version), points, and clicks on "do it" within seconds
Your setup One app at a time A whole workspace in one sentence
Your information Mostly its own apps Your files, Gmail, notes and calendar, with sources
Where it lives One computer Alexa+, the assistant you already use at home, now at your desk
Who can extend it The platform owner Anyone: it's an open MCP server

Status: with limited time, some features are implemented but not fully tested yet. Everything in the demo runs for real on a Mac. The Windows agent and desk presence with a real phone camera still need real-world testing.

Safe by design: a fixed list of allowed actions, no shell commands, no deleting files, no risky menu items clicked by voice, and every change shown for approval first.

How we built it

You speak ─▶ Mac menu-bar app (Whisper, local) or phone web app
               │ POST /api/voice
               ▼
     Simulated Alexa+: Bedrock agent (Claude Haiku 4.5) = MCP client
               │ initialize · tools/list · tools/call  (Streamable HTTP, spec 2025-11-25)
               ▼
     EchoDock MCP server (/mcp, 30 tools) ── Amazon Bedrock
               │ WebSocket (the laptop dials out: no port forwarding)
               ▼
     Desktop agent: menus, arrow, files, apps, workspaces ─▶ spoken reply + cards
  • MCP server: official Python SDK, stateless Streamable HTTP, spec 2025-11-25, bearer-token auth. Tools return a short spoken line and push cards and arrows to the screens.
  • Simulated Alexa+: Alexa+ for Builders is partner-only, so we took the track's simulated path. Our Bedrock agent calls the server exactly as an Alexa+ add-on would, so real Alexa+ can plug in with no server changes.
  • Amazon Bedrock: Claude Haiku 4.5 picks the tool for each request, with prompt caching (91% of each prompt comes from the cache). Nova 2 Lite handles small jobs. Titan Text Embeddings v2 powers search.
  • RAG: hybrid search in one SQLite file (sqlite-vec plus FTS5, merged with reciprocal rank fusion). Answers come only from the excerpts it found.
  • Desktop agent: Python + PyObjC on the macOS Accessibility API, with a click-through overlay for the arrow. A Windows version is written but not yet tested on a real PC.
  • Phone app: plain HTML/JS with MediaPipe face detection in the browser, hosted on Amazon S3.
  • Measured, not guessed: 130 test phrases, including misheard words, vague requests and 8 safety traps. A search eval checks retrieval. Every live request is traced: time, tokens and cost per step.

Challenges we ran into

  • Picking the right menu item. VS Code has 300+ menu items. A cheaper model kept answering "open the terminal" with Split Terminal. I built tricky test questions and switched models until all of them passed. Exact matches now skip the AI entirely.
  • Making "do it" mean something. On its own, it's a meaningless request. The server remembers what it last pointed at.
  • macOS permissions. They went to Terminal instead of Dock and broke after restarts. A small launcher app gives one clear permission to grant.
  • Search that quietly missed things. Bedrock throttled 801 embedding calls, so most files were missing from the index. Retries with backoff fixed it, and a full rebuild had nothing missing.
  • Honest safety. The agent once offered to "delete those files," which it can't do. Now every safety test gets an honest no plus the closest safe option.

Accomplishments we're proud of

  • Help inside any Mac app with menus: spoken, pointed at, and done on "do it."
  • 130/130 test phrases routed correctly, all 8 safety traps included, at 0.89 s median and ~$0.0018 per request.
  • 22/22 search answers correct with the right source, and 3/3 honest "not found" answers.
  • A full desk assistant for about $3/month that uses about 190 MB of RAM.

What we learned

  • MCP is the right boundary. Build the tools once, and any assistant can use them.
  • Evals change the product. They caught regressions and unsafe replies that manual testing missed.
  • Short answers win. Speak one sentence, and show the rest on screen.
  • The best help stays out of the way. Find it, open it, point at it. Don't take over.

What's next for EchoDock

  • Real Alexa+: register as an add-on as soon as builder access opens, so the Echo Pop calls the same tools.
  • Testing: finish real-world testing of the features built but not yet fully tested.
  • Windows: test and ship the agent that's already written.
  • Proactive help: heads-ups before deadlines, and notifications held during focus.
  • Teams: shared workspaces, so a new teammate can ask "where's the export for our weekly report?" and be shown.

Built With

Share this project:

Updates

Submission history