KiroWhisper

Manage your Kiro Crew agents by voice — hands full, over coffee. KiroWhisper handles the safe approvals and speaks up only when you're truly needed. Live on a real Echo Dot Max, not a simulation.

Comment

Inspiration

It started with my own daily annoyance. I often run three projects at once, and Kiro kept stopping to ask me: "create this file?", "run this migration?", "delete these lines?" — dozens of times an hour. Turning back to the screen to answer each one, across three projects, was my single biggest interruption. The agent that was supposed to free me kept pulling me back.

This is the next step in a direction I've been building toward. My earlier project, KiroCan, lets you drive the Kiro IDE from a physical Logitech MX keypad — it's live on the Logi Plugin Marketplace (https://marketplace.logi.com/plugin/KiroMxConsole/en). That was hands-on control while you sit at the machine. KiroWhisper is the same idea, evolved: voice control of an autonomous Kiro Crew for when you're not at the machine at all. IDE-at-your-desk became crew-across-the-room.

So picture the better morning. You've just woken up, coffee in hand, at the kitchen table. Your Echo is right there. You get curious — "Alexa, ask kiro whisper what my crew is up to" — and it tells you: it passed a few safe steps, one task failed, one decision is waiting. You hear it out, ask "why is that risky," say "approve it," "yes" — and go back to your coffee. You never opened the laptop. That calm oversight from across the room is the everyday magic we were after.

And then there's the deeper reason. For a blind or low-vision developer, this isn't a convenience — it's the difference between overseeing an autonomous agent and not being able to at all. A sighted developer skims twenty approvals on screen in five seconds; by voice, twenty approvals are twenty interruptions, and stepping through them one by one is simply unusable. So accessibility here isn't a label we added at the end — it's the constraint that forced the right design, and it carries real social impact: it lets a developer who can't see the screen supervise the same autonomous agents as everyone else, independently. The same thing that makes my coffee-in-hand morning pleasant makes their work possible. That's the curb-cut effect — build it for the person who needs it most, and everyone benefits.

What it does

KiroWhisper lets you oversee an autonomous Kiro Crew by voice, through Alexa, without looking at a screen. The framing that drives every design choice: it's a colleague, not a command bot. Most voice assistants are tools — you give an order, they obey once ("turn on the lights"). KiroWhisper is the teammate working in the next room who gets on with the job, consults you when it matters, and proceeds safely when it doesn't: it keeps state across the conversation, talks back and forth, and asks before doing anything risky. That's what moves it from "a basic MCP wrapper" to a context-aware, agentic add-on.

Its heart is Ambient Awareness, a filter that turns noise into signal. It watches every approval the crew raises and sorts each into one of three buckets: (1) silently pass rule-certain low-risk actions (reads, listings) once you trust the session; (2) silently block catastrophic ones that match an explicit deny rule; (3) surface — by voice — only what genuinely needs a human, highest risk first, read aloud.

Then you hold a real, multi-turn conversation with your crew, all eyes-free: "What can you do?" gives a spoken menu (a blind user can't scan the tools); "Get status" gives a "while you were away" digest of what it passed, blocked, and what waits; "Which task failed?" / "Tell me about that session" rolls up your sessions and describes the most relevant one so you can dig in without a screen; and "What needs my decision?" → "Why is that risky?" → "Approve it" → "yes." Every approval takes two confirmations — a single stray "yes" can never delete anything, and that safety is deterministic, not left to how a language model phrases a turn.

How we built it

We built KiroWhisper with Kiro, spec-driven. The whole project was written in the Kiro IDE using its spec-driven workflow — requirements → design → tasks before code — and those specs are committed in the repo at .kiro/specs/kirowhisper/ (requirements.md, design.md, tasks.md). Kiro wasn't just the editor; it was the development method, and the specs are the paper trail from "what a blind developer needs" to the tools we shipped.

The product itself is a real, self-hosted MCP server (spec 2025-11-25, over Streamable HTTP) with 13 voice-oriented tools. It's not a thin wrapper: the tools drive the Kiro Crew Gateway (read status, list pending approvals, approve/reject) and Amazon Bedrock at runtime — the track's required tech is actually called, not just named. Ambient Awareness runs deterministic rules first (fast, testable, tighten-only) and calls Bedrock (Converse / Nova) only for the ambiguous cases, returning UNKNOWN on any doubt so nothing dangerous is ever silently downgraded; Bedrock also turns raw diffs and status into short, voice-friendly speech, always with a deterministic fallback so the voice never goes silent.

For the Alexa path, an Alexa Skill hands each spoken turn to an agent on Amazon Bedrock AgentCore Runtime. That agent comes from Kay Lerch's open-source alexa-skill-mcp-bridge (https://github.com/KayLerch/alexa-skill-mcp-bridge), built on the Strands SDK + Nova; it calls our MCP tools and speaks the result. Since the official Alexa+ MCP add-on toolkit isn't public yet, this bridge is the transport that lets a self-hosted MCP server reach a real Echo — the intelligence (Kiro Crew oversight, risk filtering, safe approval) is entirely in our own MCP server. We used the bridge as-is and contributed fixes back to it during the hackathon (our Open Source mini-challenge). We verified the whole thing on a real Echo Dot Max, end to end: the two-step approval actually deleted a file, confirmed in our audit log. 157 automated tests back it, and every decision is logged (responsible-AI, and the source of the "while you were away" digest).

We did not take the simulated route. The rules let entrants submit a simulated Alexa+ experience — a mock UI, no real agent, no voice. We built the real thing instead: a genuine self-hosted MCP server (spec 2025-11-25, Streamable HTTP) that an actual Alexa Skill reaches on a physical Echo Dot Max, with the risk filtering and two-step approval running live against a real Kiro Crew. The file deletion in the demo is a real action on the real device, not a scripted mock — the audit log is the proof.

Challenges we ran into

The hard part was never capability — it was the seam between a deterministic MCP server and a generative, long-lived, tunnelled Alexa+ voice agent. A short local tool call and a real Echo conversation behave nothing alike, and almost every challenge lived in that gap. We kept a detailed friction log — 24 entries (F1–F24), each with steps, expected vs. actual, severity, workaround, and a suggestion. The ones that shaped the design, in Alexa+ terms:

The MCP session died between turns. Behind a tunnel with minutes of idle time, the next voice turn failed with -32600 "Session not found" — the tunnel drops idle SSE streams and the client kept a dead session id. Fix: run the MCP server stateless (stateless_http, json_response) so every call stands alone, plus a reconnect-and-retry in the bridge (F11).

A plain-text tool still advertised a schema, and it broke speech. A -> str tool auto-advertises an output schema and returns structuredContent; the Alexa bridge then fed the model only that JSON and dropped the spoken text, so it summarized our decision instead of reading it aloud — unusable for an eyes-free user. Fix: structured_output=False, so the reply is one plain block of words to speak (F14).

The voice model rewrites your words — on the safety-critical turn. On the real Echo, our confirm question ("…are you sure?") was paraphrased into a statement ("…please say yes to go ahead."), dropping the "?" so the mic closed and the user's "yes" was lost. Fix: give the model nothing to reword — one bare question it must speak verbatim (F21). Even after that, a correctly worded confirmation phrased as a statement ("please say yes to confirm.") still closed the mic, because the bridge decided "keep listening" purely from a trailing "?". We widened that rule in our bridge fork to recognize a spoken yes/no prompt, so a confirmation keeps the mic open even without a question mark (F24).

A new tool is invisible until the runtime is recreated. The AgentCore agent snapshots the tool list once per warm session and the container outlives redeploys, so a shipped tool stayed unknown through regenerate, redeploy, restart, and env changes — only recreating the runtime worked. Adding a voice tool is a four-step deploy, not one (F17).

Never trust the spoken reply as proof. Alexa+ sometimes answered for the skill (no Lambda invocation at all); a spoken count drifted because the model relayed a number from an earlier turn. The lesson, over and over: keep the safety-critical logic deterministic and verify the real effect at the source, never in the words (F10, F13).

The long-lived server is its own failure class. A server left running overnight kept a Gateway token it had fetched once at startup; the token expired and every call quietly turned into a 403, so a tool "succeeded" but the voice answer carried a "forbidden" error mid-rehearsal. The fix is the same discipline: refresh the token on demand rather than trusting one startup fetch (F23).

Accomplishments that we're proud of

We built the real thing, not a simulation. The rules allow a simulated Alexa+ experience — a mock interface, no real agent, no voice — and a judge couldn't always tell a mock from the real device. We didn't take that route. KiroWhisper is a genuine self-hosted MCP server (spec 2025-11-25, Streamable HTTP) that an actual Alexa Skill reaches on a real Echo Dot Max, working end to end — including the two-step approval actually deleting a file, confirmed in the audit log. On that device we chained a full eyes-free conversation across 13 MCP tools in one open session — "what can you do" → "get status" → "which task failed" → "tell me about that session" → "what needs my decision" → "why is that risky" → "approve it" → "yes" — the multi-turn, context-aware oversight the rubric calls creative.

Ambient Awareness makes voice oversight of an autonomous agent genuinely usable — the difference between "twenty interruptions" and "the one decision that matters." Most voice-agent demos ask you to approve everything by voice; we filter so you only hear what matters. We also treated accessibility as architecture: the blind-developer constraint produced a better product for everyone (the curb-cut effect). And the safety model is defense-in-depth — the two-step approval and the three-bucket filter hold regardless of how the language model phrases a turn, backed by 157 automated tests that map to the specs.

Two more we're proud of: the project was built end-to-end in Kiro with a spec-driven workflow (committed requirements → design → tasks, in .kiro/specs/), and the 24-entry friction log is real R&D — every seam between a deterministic MCP server and a generative voice agent, each with a concrete, shipped fix — not a formality.

What we learned

Deterministic core, model at the edges. When a generative model sits between your tool and the speaker, don't trust it to preserve wording, keep a turn open, or relay a value. Make the safety-critical behavior deterministic; use the model for nuance and phrasing, always with a fallback.

Voice is not a screen. You can't scan it, so the win isn't more tools — it's filtering to the few decisions that matter and keeping one conversation flowing.

The Amazon ecosystem fits this shape well. Kiro Crew's human-in-the-loop approvals are the perfect primitive, Bedrock's Converse API is model-agnostic and cheap enough to poll, and MCP over Streamable HTTP is exactly what the Alexa+ track needs.

What's next for KiroWhisper

Proactive voice ("KiroWhisper: task 3 needs you") once Alexa+ opens a proactive channel to add-ons. A glanceable decision card on screen-equipped Echo devices — voice-first, screen-optional. Clarifying dialogue — the agent asks you an open-ended question and you answer by voice. Budget awareness by voice — "how much have we spent today?" — once the Gateway exposes credit/token usage (our Feature Request FR1). And beyond one developer: shared oversight of a team's agents, still eyes-free.

Feature Requests

Full detail: https://github.com/memetcircus/kirowhisper/blob/main/docs/PRODUCT_FEEDBACK.md FR1 — expose per-account credit/quota and cumulative token usage via the Kiro Crew Gateway API. Why it matters: an eyes-free supervisor can't see the IDE's credit bar, so without this KiroWhisper cannot honestly answer "how much have we spent today?" by voice. Priority: Important.

Friction Log

Full log: https://github.com/memetcircus/kirowhisper/blob/main/docs/FRICTION_LOG.md 24 entries (F1–F24) + a severity tally, spanning the Kiro Crew Gateway, Amazon Bedrock, the MCP Python SDK, the Web Speech API, and Bedrock AgentCore — each with the task attempted, steps taken, expected vs. actual result, severity, the workaround used, and an actionable suggestion.

Built With

  • accessibility
  • agentic-ai
  • alexa
  • alexa+
  • amazon-bedrock
  • amazon-nova
  • amazon-web-services
  • aws-lambda
  • bedrock-agentcore
  • boto3
  • cloudflared
  • echodotmax
  • httpx
  • kiro
  • kiro-crew
  • mcp
  • model-context-protocol
  • pytest
  • python
  • spec-driven-development
  • starlette
  • strands
  • streamable-http
  • typescript
  • voice-ai
Share this project:

Updates

Submission history