Respondr

Respondr is an experimental application intended to assist users in recognizing potential overdose indicators and in preparing clearer information for emergency services.

Comment

Inspiration

During the rise of the opioid epidemic, we were struck by how often people walk past someone in distress - unsure whether that person is asleep, intoxicated, or overdosing.
In 2025 alone, over 77,000 overdose deaths were recorded in the U.S., with an estimated 816,000 people addicted to fentanyl. We wanted to explore how AI could empower ordinary bystanders to make faster, better-informed decisions - bridging the gap between uncertainty and action during those critical moments before help arrives.

What it does

Respondr is a Progressive Web App (PWA) designed to help identify possible overdose situations using a short 10-second video recording.
It performs a local capture, uploads the video to an API, and uses Amazon Bedrock (Nova) and Bedrock Knowledge Bases to:

  • Assess the likelihood of an overdose based on visible and auditory cues.
  • Retrieve grounded, medical guidance from trusted health resources.
  • Provide structured, human-readable steps for 911 operators.
  • Generate a concise SMS message to 911 (where available), prefilled with the assessment summary, GPS location, and a secure link to the incident report.

It’s designed for simplicity, speed, and accessibility across devices.

How we built it

Architecture overview:

  • Frontend: Built with Vite + React + Tailwind, featuring video capture, file compression, and a clean tab-based layout.
  • Backend: A serverless Node.js API deployed via AWS App Runner, provisioned entirely with Terraform, and integrated with:
    • Amazon Bedrock Nova for multimodal inference (image + text)
    • Amazon Bedrock Knowledge Base using S3 Vector Store for retrieval-augmented grounding
    • DynamoDB for deduplication and metadata
    • Amazon S3 for secure video and content storage
  • Crawler subsystem:
    A scheduled Node.js Lambda crawler retrieves the latest overdose-related and emergency-response literature from the PubMed API.
    Each retrieved article is normalized and uploaded to an S3 bucket, which triggers a Knowledge Base sync job. The sync process embeds and indexes the documents into the S3 Vector Store, making them immediately searchable by Bedrock Nova.

This creates a self-updating AI knowledge layer grounded in verified medical sources, ensuring Respondr’s guidance remains relevant and evidence-based.

Challenges we ran into

  • Provisioning Knowledge base in combination with S3 Vector via Terraform
  • Cross-browser video handling: ensuring consistent recording, compression, and format compatibility between Chrome, Safari, and mobile browsers.
  • Bedrock Knowledge Base ingestion: building a stable PubMed crawler and sync mechanism within API limits.
  • Latency tradeoffs: balancing video quality with upload and inference speed.
  • Ethical UX: designing a user experience that supports decision-making without implying medical authority.
  • Building a solution that protects the privacy of both the subject and the reporter.

Accomplishments that we're proud of

  • Built a functional end-to-end AI incident assistant powered by Bedrock Nova and Knowledge Bases.
  • Implemented an automated PubMed ingestion pipeline that continuously grounds AI outputs in current medical research.
  • Designed a clean, responsive PWA interface with guided video capture and automated 911 message / call preparation.
  • Deployed a complete solution with Terraform.
  • Demonstrated how AI can responsibly assist in emergency awareness and response without replacing human judgment.

What we learned

  • How to integrate multimodal AI (video + text) with Amazon Bedrock.
  • The practical benefits of RAG (Retrieval-Augmented Generation) and S3 vector embeddings for grounding AI responses.
  • How to architect self-updating knowledge systems using S3 events and Lambda triggers.
  • Lessons in browser media APIs, cross-format handling, and service communication patterns.
  • The importance of designing AI systems that enhance confidence and clarity—not automation.

What's next for Respondr

  • Feedback loop - develop a feedback loop to verify assessments and continuously refine the model for improved accuracy.
  • Build a native mobile app for iOS and Android to provide a smoother, faster, and more reliable user experience with deeper access to device sensors and offline capabilities.
  • Develop a research API that can securely share anonymized assessment data and incident patterns with approved public health and research institutions, helping generate valuable insights into overdose trends and early intervention effectiveness.
  • Pursue public funding and health innovation grants to support continued development, compliance review, and large-scale deployment - bringing Respondr to the broader public as an accessible, evidence-based safety tool.
  • Continue refining the AI assessment pipeline, improving accuracy, and expanding the medical knowledge base with new verified data sources such as CDC and WHO publications.
  • Strengthen privacy and ethical governance frameworks to ensure that every improvement continues to protect both the subject and the reporter.
Share this project:

Updates