SwiftAid- Emergency AI Assistant

Meet Swift Aid: Streaming emergency calls into our AI engine, we give callers instant, spoken safety steps and auto-alert first responders, because every second counts.

Comment

🚨 Inspiration

SwiftAid is a serverless, multi-turn AI-powered emergency response system built on AWS. It streams live audio from Amazon Connect, transcribes and analyzes it in real time using Amazon Transcribe and Amazon Bedrock (Claude v2), and responds with human-like guidance via Amazon Polly, all within a single phone call.

🌟 Why We Built SwiftAid

Every Second Matters: In a crisis, delays can cost lives. SwiftAid steps in to provide immediate, clear instructions while human dispatchers gear up.

Augment, Don’t Replace: Rather than taking over, SwiftAid empowers first responders by pre-triaging calls, identifying urgent needs, and automatically notifying the right teams.

Emotional Support: Panic and fear can cloud judgment. Our AI listens for vocal cues, senses stress levels, and adjusts its tone to keep callers calm.

💡 Core Features & AWS Services

Capability AWS Service
Contact management Amazon Connect
Real-time audio streaming Amazon Kinesis Data Streams
Streaming transcription Amazon Transcribe Streaming
Sentiment Analysis Amazon Comprehend.
Conversation state storage Amazon DynamoDB
Generative AI response Amazon Bedrock (Claude v2)
Speech synthesis Amazon Polly
Attribute updates Amazon Connect API
Logging & monitoring Amazon CloudWatch & X-Ray

🤖 Generative AI & Emotion Detection

Claude v2 on Bedrock powers SwiftAid’s natural language understanding and generation. Prompt engineering ensures:

  • Empathy: The AI chooses words that soothe anxiety.
  • Clarity: Action steps are concise, direct, and unambiguous.

Voice Analytics: By analyzing speech patterns (pitch, volume, pace), SwiftAid detects stress or panic, adapting its language to reassure callers.

Real-Time Loop: The multi-turn exchange feels conversational, SwiftAid asks follow-ups, confirms details, and remains on the line as a virtual companion.

💥 Impact & Benefits

  • Faster Dispatch: Automates the initial triage, cutting precious seconds off response times.
  • Reduced Call Overload: Handles routine data gathering, address, injury status, immediate risks, freeing human operators for complex scenarios.
  • Improved Outcomes: Immediate first-aid guidance can prevent escalation before EMTs arrive.
  • Data Insights: Every interaction logs time-to-response, sentiment scores, and key medical or safety terms, enabling continuous improvement and analytics.

📞 How It Feels for Callers

I was terrified when the accident happened, but the AI’s calm voice walked me through every step. I felt heard and guided, even before sirens were in earshot.

Callers experience:

  • Immediate reassurance (“Help is on the way”)
  • Actionable steps (“Apply gentle pressure to your wound”)
  • Continuous support until help arrives.

💡 What it does

SwiftAid’s core begins in Amazon Connect, where a custom contact flow greets callers, captures their speech in real time, and manages the multi-turn loop, all without ever leaving the phone line. When a user speaks, Connect’s media streaming blocks funnel PCM audio directly into a Kinesis Data Stream, triggering the next step in our pipeline.

At Amazon Kinesis Data Streams, those raw audio chunks are ingested and ordered by contact, ensuring no piece of the conversation is lost or out of sequence. From there, a dedicated Lambda consumer decodes each record and seamlessly feeds it to the transcription engine.

Using Amazon Transcribe Streaming, SwiftAid transforms speech into text with sub-second latency. Only complete, finalized transcripts are passed on, filtering out partial or speculative phrases to maintain clarity and accuracy in high-stress situations.

The transcripts and metadata are then stored in Amazon DynamoDB, providing a durable, queryable history of the full conversation. DynamoDB Streams fire as new user entries arrive, automatically invoking our orchestration logic to generate intelligent responses.

Our orchestration lives in Amazon Bedrock (Claude v2), where a carefully crafted system prompt ensures responses are calm, concise, and action-oriented. The AI analyzes the full dialogue context, then returns precise guidance tailored to the emergency at hand.

Once the text response is ready, Amazon Polly converts it into a natural-sounding MP3 using a neural voice. That audio file is stored in S3 and made immediately available for playback.

By setting a custom contact attribute via the Amazon Connect API, SwiftAid delivers the Polly-generated MP3 back into the ongoing call, closing the loop and letting the caller hear life-saving instructions in real time.

Behind the scenes, Amazon CloudWatch and AWS X‑Ray provide deep observability, tracing each step from stream ingestion to AI execution so we can monitor performance, troubleshoot issues, and keep latency under strict targets.

This forms a real-time AI-human interaction loop during high-stress emergencies, ensuring no moment is wasted waiting for human dispatchers.

🛠️ How we built it

Serverless & Infrastructure as Code: Provisioned all AWS resources (Connect, Kinesis, Lambdas, DynamoDB, S3) via AWS Console and GitHub Actions-driven IaC for reproducibility.

  • 📞 Amazon Connect receives emergency voice calls and streams them live via Amazon Kinesis Video Streams. Designed a bespoke multi-turn flow with media streaming, polling for AI responses, dynamic prompt playback, and error-safe looping for uninterrupted calls.

  • 🧠 A Lambda function consumes the stream, buffers audio chunks, and launches a transcription job using Amazon Transcribe.

    • Swiftresponse Lambda: 🗂️ Deployed a Transcribe Streaming SDK layer to decode Kinesis payloads, perform sub-second transcription, and write finalized text segments into DynamoDB.
    • SwiftAIOrchestrator Lambda: 🤖 Triggered by DynamoDB Streams to assemble conversation history, invoke Claude v2 on Amazon Bedrock, synthesize responses via Amazon Polly 🗣️, upload MP3s to S3, and push audio URIs back into the live Connect session.
    • ClearAudioUri Lambda: Integrated a lightweight reset function in the Connect flow to clear the custom audio attribute between turns, enabling seamless repeated interactions.

  • Kinesis Data Stream: Established SwiftAI-AudioStream with ContactId partitioning to reliably ingest and order caller PCM audio in real time.

  • Observability & Monitoring: Activated detailed CloudWatch Logs and AWS X-Ray tracing across all services, providing end-to-end visibility and ensuring consistent sub-8-second response latency.

The entire workflow is serverless, event-driven, and built for real-time response.

⚠️ Challenges we ran into

  • Maintaining ultra-low latency from voice input to AI voice output (~6–8 seconds end-to-end).
  • Ensuring audio buffers were stable before triggering transcription jobs via Lambda.
  • Designing robust Claude prompts to balance empathy with clarity during medical crises.
  • Managing Kinesis stream consumption and Lambda concurrency, especially under high call volume.
  • Orchestrating real-time DynamoDB updates while multiple Lambda invocations modified the same record.

🏆 Accomplishments that we're proud of

  • Built a fully operational, real-time serverless emergency AI assistant powered by 8 AWS services, including AWS Bedrock and AWS Q.
  • Delivered natural-sounding, helpful AI voice feedback to real emergency voice input, without any app or interface.
  • Developed robust prompt designs for Claude v2 that balanced empathy and clarity under high-stress scenarios.
  • Achieved < 8-second end-to-end latency from call audio to AI voice response, ensuring timely guidance.

📚 What we learned

  • Mastered streaming pipelines by integrating Connect → Kinesis → Transcribe → DynamoDB → Bedrock → Polly.
  • Engineering real-world Claude prompts that help save lives, not just answer trivia.
  • Using DynamoDB as a central orchestrator for Lambda-based pipelines.
  • Optimizing AWS Lambda for complex, stateful real-time use cases beyond the typical REST patterns.
  • Learnt to deploy and manage Lambda layers for native dependencies like AWS CRT, enabling sub-second transcription.
  • Discovered the importance of looping logic and state management in connect flows to maintain conversation continuity.

🌟 What's next for SwiftAid

  • 🌍 Add multilingual transcription + voice playback via Amazon Translate and Polly's voice library.
  • 🚑 Integrate EMT and 911 dispatcher alerts via SMS/Push for critical cases.
  • 📊 Build a real-time analytics dashboard for emergency supervisors to monitor urgency, keywords, and stress levels.
  • 🏢 Package SwiftAid as a SaaS platform for hospitals, public safety orgs, and smart cities.
  • 🧠 Integrate continuous learning models that refine responses based on feedback loops from real calls.

Built With

Share this project:

Updates