OmniPulse

Inspiration

Most voice assistants operate as stateless, single-turn command executors—forcing users to repeat household context, health needs, and preferences across separate apps. With the launch of Alexa+ and open Model Context Protocol (MCP) standards, we wanted to build an ambient home experience for the Amazon Echo Show 15 that remembers who lives in the home, anticipates visiting family needs, and orchestrates Smart Home, Wellness Commerce, and Ring Security in a single multi-service turn.

What it does

Alexa+ OmniPulse is an autonomous, context-aware ambient orchestrator powered by MCP Specification 2025-11-25 over Streamable HTTP and an Alexa+ Agent Skill (SKILL.md):

Memory-First Ambient Invariant: OmniPulse never blindly executes commands. Every interaction first calls get_household_memory to retrieve persistent cross-session SQLite context (e.g., "Parents visiting tomorrow; father has knee pain, prefers low-sodium Ayurvedic wellness items, and warm 2700K ambient lighting").
Autonomous Multi-Service Orchestration: A single prompt like "Prep home for visiting parents and find knee care items" autonomously chains update_smarthome_state (setting living room lights to 2700K warm relax mode and HVAC to 72°F) and search_and_build_carousel (filtering in-stock low-sodium Ayurvedic joint support items) in a single turn.
Rich Visual Media & 1-Click Voice/Touch Checkout: Returns structured UI card payloads alongside natural spoken dialogue to render interactive Product Carousels, 1-Click Instant Checkout Cards (execute_instant_checkout), Smart Home Control Tiles, and Ring Security Alerts.
Proactive Cross-Device Ring Doorbell Trigger: When the Ring webhook (POST /webhook/ring) detects a visitor or package at the front porch, handle_ring_event cross-references the visitor against household memory, verifies visiting parents, automatically unlocks the front door, and pushes a live alert card to the Echo Show 15 display via Server-Sent Events (GET /events).
Live MCP Streamable HTTP Inspector: Includes a built-in protocol inspector drawer in the simulator that executes and logs real-time JSON-RPC 2.0 initialize, tools/list, and tools/call traffic to prove runtime MCP 2025-11-25 compliance.

How we built it

Spec-Driven Engineering with Kiro IDE (AWS Builder): Used Kiro's spec workflow (.kiro/steering/ and .kiro/specs/omnipulse-mvp/) to architect our 3-layer system design, define 10 formal correctness properties, and orchestrate implementation tasks.
MCP Server & Backend (/backend): Built with Python 3.13, uv, FastAPI, and SQLite (omnipulse.db in WAL mode). Implemented a compliant Streamable HTTP MCP Server (POST /mcp) supporting JSON-RPC 2.0 initialize (protocolVersion: "2025-11-25"), tools/list, and tools/call across 6 domain tools (get_household_memory, save_household_memory, update_smarthome_state, search_and_build_carousel, execute_instant_checkout, handle_ring_event), plus an SSE event stream (GET /events).
Zero-Key Smart Fallback + LLM Bridge: Built a deterministic multi-domain fallback engine so hackathon judges can run the entire stack out-of-the-box with zero external API keys, while supporting optional AWS Bedrock / Gemini / OpenAI credentials.
Echo Show 15 Simulator (/frontend): Built with React 19, TypeScript, Vite, Tailwind CSS v4, Lucide Icons, and the Web Speech API (STT/TTS) for hands-free voice interaction and animated audio frequency visualization.

Challenges we ran into

Gated Alexa+ Preview Tooling: Because the official Alexa+ Web Simulator and Category SDK are restricted to select preview partners, we engineered our own Echo Show 15 Web Simulator with a live MCP Streamable HTTP Inspector to transparently demonstrate real JSON-RPC 2.0 protocol handshakes and tool executions.
Standardizing Multimodal UI Cards over MCP: Standard MCP tools/call responses return text/resource content blocks but lack a canonical schema for interactive Echo Show APL cards. We extended our tool execution envelopes with a strongly typed ui card contract (carousel, smarthome_card, ring_alert_card, checkout_card).

Accomplishments that we're proud of

100% Automated Verification (38 Passing Tests): Verified all 10 formal correctness properties (protocol version compliance, memory-first invariant, tool count = 6, checkout math accuracy, in-stock filtering, smart home clamping, and idempotent DB seeding) using pytest and property-based testing (38 passed in 3.53s).
True Multi-Device Synergy: Bridged Alexa+ conversational commerce, Smart Home climate/lighting, and proactive Ring doorbell security into one cohesive ambient loop.

What's next for Alexa+ OmniPulse

Deploying our Streamable HTTP MCP Server onto AWS Lambda + Amazon Bedrock AgentCore for serverless production scaling.
Adding Fire TV cross-screen handoff so Ring camera feeds and wellness guides can cast directly to the living room TV.