End-to-end architecture of AllSenses AI Guardian: real-time audio capture, AI threat detection, and AWS-powered emergency response.
Overview: End-to-end architecture of AllSenses AI Guardian: real-time audio capture, AI threat detection, and AWS-powered emergency response
Block Diagram:End-to-end architecture of AllSenses AI Guardian: real-time audio capture, AI threat detection, AWS-powered emergency response

Perfect — here’s a new, improved version of your “Project Story” (fully Markdown-formatted and ready to paste directly into the Devpost → About the project box). It keeps the narrative power of your previous draft but updates it to match your latest build (with real SMS + geolocation + front-end sequence) — and flows naturally for hackathon jurors:

🛰️ AllSenseAI — The AI Guardian That Hears Before It’s Too Late

🌍 Inspiration

Every day, millions face moments when calling for help is impossible — while commuting, walking home, or simply working late. Investigations like The New York Times exposé on rideshare assaults revealed how safety can collapse when systems fail to detect distress or communicate it in time.

AllSenseAI was born from that reality — to create an AI-driven guardian that perceives danger through sound, verifies it intelligently, and triggers the right response automatically.

Reference: The New York Times – Uber’s Sexual Assault Problem

⚙️ What it does

AllSenseAI is a real-time AI guardian that listens, understands, and acts.

Using the device microphone, it continuously interprets environmental sounds and spoken cues through Amazon Bedrock models trained for emotional tone and distress recognition. When danger is detected, it launches a 7-step emergency flow — powered entirely by AWS:

Audio Capture – The app listens through the microphone.
Distress Detection – Bedrock analyzes sound patterns and keywords related to danger.
Event Trigger – Once confidence passes threshold, AWS Lambda sends an emergency_triggered event via Amazon EventBridge.
Geolocation Retrieval – Amazon Location Service reverse-geocodes GPS coordinates into an exact address.
SMS Dispatch – Amazon Pinpoint sends a real SMS:

“🚨 Emergency alert for {victimName}. Possible danger detected at {placeName}. [Map Link]”

Contact Confirmation – If any contact replies “OK,” the alert closes; otherwise it escalates to a secondary channel (voice or push).

Analytics & Learning – Amazon CloudWatch + X-Ray anonymize logs to improve future detection accuracy.

The entire sequence is visualized live on the front-end “Incident Panel,” where each step lights up as it executes — showing jurors how AI, data, and automation converge to save seconds that can save lives.

🧠 How we built it

With KIRO, our autonomous AI developer, we built AllSenseAI as a modular serverless system orchestrated on AWS:

Amazon Transcribe Streaming – captures live audio and produces text streams.
Amazon Bedrock (Claude 3.5 / Nova Micro) – interprets emotion and context.
AWS Lambda + EventBridge – handle real-time event routing.
Amazon Location Service – provides precise map and address data.
Amazon Pinpoint / SNS – deliver real SMS alerts with two-way confirmation.
Amazon DynamoDB – stores user profiles and trusted contacts.
AWS KMS – encrypts all personal and geolocation data.
CloudWatch / X-Ray – monitor latency, delivery, and model performance.
Deployed through AWS SAM for consistent, reproducible infrastructure.

🧩 Challenges we faced

Detecting emotional distress accurately in noisy real-world environments.
Balancing sensitivity vs. false positives — every false alarm erodes trust.
Coordinating multiple AWS services within sub-second latency.
Handling encrypted data while respecting privacy and regulatory compliance.
Designing an interface simple enough for jurors to see complex AI logic in action.

🏆 Accomplishments we’re proud of

Building the first AWS-native pipeline that captures real audio, verifies distress, and sends verified SMS alerts including the victim’s name and live location.
Achieving < 5 seconds end-to-end response from detection to delivered SMS.
Demonstrating transparent AI safety logic through a live 7-step visual panel.
Implementing full encryption, audit trails, and consent-based operation.

AllSenseAI proves that empathy and technology can coexist — transforming awareness into immediate action.

💡 What we learned

We learned that AI’s greatest role isn’t replacement — it’s extension: to hear when someone cannot speak and act when every second matters.

We deepened our mastery of AWS AI/ML services, event-driven design, and responsible AI principles. Most importantly, we learned that real-time protection depends as much on trust and privacy as it does on algorithms.

🚀 What’s next

Integration with wearables and smart glasses for continuous ambient awareness.
Specialized detection modes for mobility, healthcare, and campus safety.
Regional compliance for 911 and global emergency routing.
A family & guardian dashboard to manage contacts, alerts, and consent.
Ongoing ML training using anonymized CloudWatch data for better accuracy.

Our vision: a universal AI guardian — silent, trusted, always present — protecting life in real time.

Demo: Demo: Demo: Demo

https://d4om8j6cvwtqd.cloudfront.net/audio-monitor.html