Inspiration
What it does
How we built it
Inspiration
SENSEGUARD AI was inspired by a simple but important observation: some emergencies happen when a person is unable to ask for help.
A person may fall, become immobile, experience distress, or be exposed to unsafe environmental conditions while nobody is nearby to notice. Conventional safety systems often depend on a single sensor, a fixed threshold, or a manual alarm. This can miss the context of what is actually happening.
We wanted to build a system that does not just detect an individual event, but understands the combination of signals and recognizes when they may indicate a developing emergency.
"When someone cannot call for help, let technology notice."
What it does
SENSEGUARD AI is a multimodal early-warning system for human safety.
It combines three types of signals:
- Vision: posture, movement, falls, and prolonged immobility
- Audio: speech, distress sounds, and unusual acoustic activity
- Environmental sensing: temperature, CO₂, air-quality-related changes, and vibration
Instead of relying on one signal, the system correlates person + environment + time to estimate a risk level and generate an explainable alert for a trusted human.
The overall pipeline is:
Sense → Understand → Correlate → Protect → Human Help
The system produces a risk score between (0) and (100\%), along with the signals that contributed to the alert.
How we built it
We designed SENSEGUARD AI as a multimodal AI pipeline.
1. Vision
For visual perception, we use YOLO, MediaPipe, and OpenCV for person detection, tracking, posture estimation, and fall/immobility analysis.
2. Acoustic intelligence
Audio is converted into spectrogram and MFCC-based features and processed using a CNN-based acoustic classification pipeline to identify distress calls, help requests, and unusual ambient sounds.
3. Environmental intelligence
Environmental measurements such as temperature, humidity, CO₂, and vibration are analyzed using anomaly-detection approaches including Isolation Forest, Autoencoder, and LSTM-based temporal analysis.
4. Temporal analysis
Emergency signals are not always obvious from a single frame or measurement. Therefore, the system tracks changes over time using sequential modeling.
5. Multimodal fusion
The extracted vision, audio, and environmental features are combined using a multimodal fusion layer with temporal context.
We formulate the risk estimation as:
$$ Risk_t = f(V_t, A_t, E_t, TemporalContext) $$
where (V_t), (A_t), and (E_t) represent visual, audio, and environmental information at time (t).
The final output is an explainable risk assessment that can trigger human intervention.
What we learned
Working on SENSEGUARD AI helped us understand that multimodal AI is not simply about combining multiple models.
The important part is understanding when different signals should reinforce each other.
For example, a sudden change in posture alone may not always indicate an emergency. However, posture change combined with prolonged immobility and an abnormal environmental signal can provide stronger contextual evidence.
We also learned about temporal modeling, anomaly detection, multimodal fusion, explainable AI, edge processing, and privacy-preserving monitoring.
Challenges we faced
One of the biggest challenges was designing a system that could combine heterogeneous data sources while maintaining meaningful temporal context.
Different modalities have different characteristics:
- Camera data is spatial and visual.
- Audio is time-dependent and frequency-based.
- Environmental sensors produce continuous numerical measurements.
Another challenge was reducing false alarms. A single unusual sensor reading should not automatically trigger an emergency response. This motivated our focus on cross-signal correlation and adaptive anomaly detection.
Privacy was another important consideration. Our design includes local processing of sensitive visual information and immediate disposal of raw frames wherever possible.
What's next
Our roadmap starts with a prototype combining ESP32 data, camera data, and basic environmental sensing.
Future development includes:
- Edge TPU acceleration
- Thermal and wearable sensor integration
- Pilot deployments in hostels, hospitals, and assisted-living environments
- Distributed safety networks with federated anomaly learning
Our goal is not to replace human care.
It is to help care arrive sooner.
Log in or sign up for Devpost to join the conversation.