Alexa+ LifeOps

Voice-first Echo Show care companion via MCP & Bedrock

Comment

Inspiration

India and globally, the aging-in-place and patient-care transition back home is incredibly fragmented. Older adults and chronic patients often struggle with manual vitals logging, confusing prescription timelines, and knowing when a symptom is a true warning versus an emergency. Voice interfaces are the most natural way for individuals to interact with technology, especially when they are unwell.

With the emergence of the Model Context Protocol (MCP 2025-11-25) and Amazon’s new developer capabilities, we saw an opportunity to transform a standard smart ambient display into an autonomous, proactive, and compassionate clinical triage assistant. We wanted to build something that doesn't just listen, but clinically evaluates, Schedules life operations, and escalates appropriately while remaining completely failsafe and highly responsive even without internet dependencies.

What it does

Alexa+ LifeOps is an intelligent, voice-first care and ambient triage assistant designed for simulated Echo Show devices. A patient can communicate naturally, and the system provides spoken guidance alongside real-time visual cards:

  1. Ambient Vitals Triage: Instantly classifies Blood Pressure, Heart Rate, and SpO2 into Normal, Warning, or Critical states using clinical heuristics, and activates a real-time, dynamic green ECG canvas visualization to capture ambient focus.
  2. Dynamic Prescription & Task Reminders: Parses conversational natural language (e.g., "Schedule my Metformin reminder tomorrow at 8 PM") to create structured task schedules and care paths.
  3. Emergency Protocol Simulation: Detects severe distress indicators (e.g., crushing chest pain) and escalates to a simulated visual emergency siren, SMS caregiver alerting, and EMS coordination. Crucially, it evaluates negation (e.g., "I do not have chest pain") correctly to avoid trigger fatigue.
  4. Instant Patient Profile Access: Retrieves Allergies, Active Medication regimens, and emergency contact registries directly on voice request.

How we built it

  • Hand-Crafted MCP Server: Built a native JSON-RPC 2.0 compliant backend in Python over FastAPI adhering to the latest MCP 2025-11-25 Streamable HTTP transport spec from scratch—completely without wrapper SDKs. Implemented endpoints for initialize (negotiation), notifications/initialized (standard HTTP 202), ping, tools/list, and tools/call.
  • Intelligent LLM Tool-Orchestration: Integrated AWS Bedrock Converse API (boto3) to map user prompts through iterative toolUse and toolResult looping in agent.py.
  • Dual-Tier Resilient Fallback: Implemented a offline deterministic clinical engine (LocalClinicalEvaluator) that shares schemas and error types with the Bedrock parser. If cloud credentials/endpoints are unreachable, it immediately falls back, reporting the engine status on-screen to prevent downtime.
  • Echo Show Display Experience: Formatted a frontend UI (static/index.html) using Browser Web Speech API for low-latency Voice input/output, real-time SVG card generators, a visual digital clock, and a custom Canvas Sinus Rhythm ECG loop triggered by vitals triage.

Challenges we ran into

  • Intricate Error Semantics: Figuring out when to return JSON-RPC standard exception objects (like -32602 for unrecognized methods) vs. valid success payloads carrying isError: true inside MCP execution scopes required tedious troubleshooting and test iterations.
  • Dynamic Argument Aliases: Aligning standard schemas to exact function signatures initially caused argument type crashes during calls (e.g. client sending patient_id when Python logic expected user_id). Resolve this by creating dynamic argument inspectors and normalizers.
  • Complex Multiturn Loops: The AWS Converse API tool config requires processing multiple toolUse payloads sequentially. Ensuring the iteration stack remains consistent without crashing or falling into feedback loops took fine-tuning.

Accomplishments that we're proud of

  • Hand-crafting a fully custom, compliant MCP Streamable HTTP stack from raw bytes without reliant wrappers that gracefully processes real-time requests.
  • Building a highly interactive, simulated device experience that doesn't just display static data but captures the urgency of clinical alerts and parses schedule times natively.
  • Creating a robust local fallback pipeline, ensuring that patient support never fails silently if Bedrock endpoints lack credentials or disconnect.

What we learned

  • How MCP acts as an outstanding, open standard for exposing structured backend services to voice workflows without complex UI bindings.
  • Designing for resilience in AI: why graceful, localized deterministic loops are mandatory for AI use-cases in human safety domains.
  • Deepened our knowledge around proper HTTP/JSON-RPC error boundaries and input schema validation patterns.

What's next for Alexa+ LifeOps

  • Hardware Integrations: Hook the vitals triage input tool up to IoT-connected blood pressure and pulse oximeter APIs instead of manual inputs.
  • Enhanced Bedrock Configuration: Move beyond demo fallback bounds once full region credential permissions are available.
  • Security Protocols: Add OAuth2 or device-bound session token auth to guard clinical histories.
  • Rich Media & APL Support: Translate screen models into real, production-ready Amazon Presentation Language (APL) payloads.

Built With

  • aws-bedrock
  • css3
  • echo-show
  • fastapi
  • html5
  • javascript-canvas
  • json-rpc
  • mcp-streamable-http
  • model-context-protocol
  • python
  • webspeech-api
Share this project:

Updates

Submission history