NovaGuard: Emergency AI for Deaf and HoH Communities

Accessible emergency intake and dispatcher decision support using all 4 Amazon Nova models, with initial triage in under 10 seconds and a full response package in under 30 seconds.

Comment

Inspiration

48 million Americans are deaf or hard-of-hearing. When they face a medical emergency, their only option is a TTY relay service that adds 3+ minutes to every 911 call. In a cardiac arrest, brain damage begins at 4 minutes. Those extra minutes aren't an inconvenience — they're the difference between life and death.

We built NovaGuard to reduce that gap. A deaf caller types their emergency. NovaGuard turns that input into a structured triage assessment, dispatch recommendation, hospital lookup, and responder briefing. In the live demo, initial triage completes in under 10 seconds and the full response package completes in under 30 seconds.

What it does

NovaGuard is a multi-agent emergency response system that:

  1. Accepts text-based emergency reports from deaf/HoH callers (no voice required)
  2. Triages with a 7-tool Strands Agent that reasons about severity, escalates critical cases, and delegates to other agents
  3. Dispatches optimal response units with real route-based ETAs via Amazon Location Service
  4. Finds nearby hospitals by automating Medicare.gov Care Compare using Nova Act on ECS Fargate
  5. Matches similar past incidents using Nova Multimodal Embeddings (3072-dim vector search)
  6. Provides voice bridging via Nova Sonic for dispatch and responder communication workflows
  7. Generates responder audio briefings using the Sonic Briefing Agent — Nova Sonic TTS produces professional spoken briefings for first responders arriving on scene
  8. Checks real-time weather at the emergency location via Open-Meteo API

The system is presented to judges as a 4-stage workflow: Triage → Dispatch Recommendation → Communications → Hospital Lookup and Responder Briefing. Behind that workflow, specialized agents communicate via A2A (Agent-to-Agent) protocol and AWS Step Functions orchestration.

NovaGuard is a decision-support prototype for emergency accessibility. It does not replace 911 professionals or make autonomous medical decisions.

How we built it

Architecture

  • 4-stage emergency workflow with specialized services behind the scenes — Intake/Triage, Dispatch Recommendation, Communications, and Hospital Lookup/Briefing
  • AWS Step Functions Express Workflow — 4-step pipeline presented consistently in the demo
  • A2A Protocol — 7 agent cards, 5 skills, inter-agent delegation
  • Strands Agents SDK — 7-tool agentic loop with conditional reasoning (not scripted)
  • MCP tools — weather hazard detection via real API
  • ECS Fargate — runs Nova Act browser automation in the cloud

Amazon Nova Models Used (ALL 4)

Model Role
Nova 2 Lite Triage, Dispatch, Communications — fast inference for critical decisions
Nova Multimodal Embeddings 3072-dimensional incident matching via cosine similarity
Nova Sonic Bidirectional STT/TTS voice bridge + Responder Audio Briefing generation
Nova Act Autonomous browser automation on Medicare.gov Care Compare

Key Technical Differentiators

  • Truly agentic: Strands Agent reasons about tool results — severity ≥85 triggers re-query, ≥90 triggers dispatch pre-alert and escalation
  • Inter-agent delegation: Triage agent invokes Dispatch agent via Lambda, not just pipeline-sequential
  • Real .gov data: Nova Act navigates Medicare.gov (a real government website) — no mocks
  • Real-time weather: Open-Meteo API checks conditions at emergency location
  • 3072-dim embeddings: Nova Multimodal Embeddings for semantic incident matching

Safety framing

  • Decision support, not autonomy: output is framed as triage and dispatch recommendation for trained personnel
  • Auditable workflow: each stage produces inspectable intermediate output instead of a single opaque answer

AWS Services

API Gateway, Lambda (13 functions), Step Functions, DynamoDB, ECS Fargate, ECR, Amazon Location Service, CloudWatch, EventBridge, S3, CloudFront, Bedrock (Nova 2 Lite, Nova Embeddings, Nova Sonic, Nova Act)

Challenges we ran into

  1. Nova Lite + Strands toolUse format: Nova 2 Lite's Converse API occasionally rejects tool result formats mid-cycle. We built a recovery mechanism that detects when all tools completed successfully but the conversation cycle fails, and constructs the result from tracked tool outputs — maintaining agent_framework: "strands-agents" instead of falling back.

  2. Nova Act on Lambda: Nova Act requires a browser runtime (Chromium), which doesn't fit in Lambda's 250MB limit. We deployed it on ECS Fargate with a custom Docker container, invoked asynchronously from Lambda.

  3. Cold start latency: The Strands Agent Lambda with 7 tools, MCP client, and Nova model initialization takes 15-20s on cold start. We added a 50s timeout guard, increased Lambda memory to 1024MB and timeout to 90s, and created a pre-warm script.

  4. Medicare.gov layout changes: The target website can change at any time. We added retry logic with simplified instructions on retry ("search for hospitals near [address]").

Accomplishments that we're proud of

  • 12/12 live API endpoints pass — verified in the current deployment
  • All 4 Amazon Nova models in production, not just referenced
  • Strands Agent calls 7 tools per triage including real weather API and inter-agent delegation
  • 24 real hospitals from Medicare.gov Care Compare — parsed and stored via Nova Act
  • A2A protocol with 7 agent cards and real inter-agent communication
  • Responder Audio Briefing — Nova Sonic TTS generates professional spoken briefings for first responders
  • CloudFront-hosted demo UI — accessible to judges at https://d10tmiea7afu9g.cloudfront.net
  • Integration test suite — test_endpoints.py validates all 4 Nova models in 28 seconds
  • Live proof endpoint — judges can verify model and service availability without opening the codebase first

What we learned

  • Strands Agents SDK is powerful but requires careful error handling around the Bedrock Converse API — tool result formatting is strict
  • Nova Act on ECS Fargate is the right pattern for browser automation in serverless architectures
  • The A2A protocol enables clean separation of agent concerns while maintaining interoperability
  • Nova Multimodal Embeddings at 3072 dimensions provide excellent semantic matching for emergency descriptions

What's next for NovaGuard

  • Production deployment with proper VPC, WAF, and HIPAA compliance
  • Multi-language support via Amazon Translate (infrastructure already wired)
  • Photo/video intake using Nova's multimodal capabilities — a deaf caller could share a photo of the scene
  • Integration with real 911 CAD systems via secure API bridges
  • Mobile app with accessibility-first design (large touch targets, haptic feedback, visual alerts)

Responsible use

NovaGuard is intended as an accessibility and decision-support layer for emergency workflows. Final operational decisions remain with trained dispatchers, responders, and emergency organizations.

Built With

  • a2a-protocol
  • amazon-dynamodb
  • amazon-ecs-fargate
  • amazon-nova
  • amazon-nova-act
  • amazon-nova-embeddings
  • amazon-nova-sonic
  • aws-bedrock
  • aws-cdk
  • aws-lambda
  • aws-step-functions
  • python
  • strands-agents
  • typescript
Share this project:

Updates