-
-
Time–frequency representation of cough audio highlighting patterns used for disease classification.
-
Interactive dashboard for symptom input, cough recording, and real-time AI-based respiratory prediction.
-
End-to-end architecture of AeroTriage showing dual-input processing (audio + symptoms) and late fusion for final prediction.
Inspiration
India faces a significant respiratory health burden, contributing to nearly 25% of global tuberculosis cases and a large share of respiratory-related disability in Asia. A major issue is the primary care gap, where most Primary Health Centers (PHCs) lack rapid diagnostic tools. Delays in diagnosis and undetected cases continue to drive transmission and mortality.
We were inspired to address this gap by building a software-driven, rapid screening system that works without expensive infrastructure. The goal was to create a tool that can assist frontline healthcare workers in identifying respiratory diseases early using accessible inputs like cough sounds and basic symptoms. :contentReference[oaicite:0]{index=0}
What it does
AeroTriage is a multi-disease diagnostic system designed to detect conditions such as COPD, pneumonia, and upper respiratory tract infections (URTI). It analyzes cough audio signals along with clinical symptom data to provide a fast and reliable triage decision.
The system focuses on identifying disease-specific patterns such as:
- Acoustic signatures in cough sounds
- Energy distribution patterns for infections
- Symptom profiles like fever, night sweats, and breathlessness
It provides a guided triage output, helping users understand their condition and next steps rather than just displaying raw scores.
How we built it
AeroTriage follows a multi-modal pipeline architecture, combining acoustic intelligence with clinical reasoning.
1. Data Acquisition
Two parallel inputs are collected:
- Acoustic Input: Cough sounds recorded in
.wavformat - Clinical Input: Binary symptom indicators (0 or 1)
2. Acoustic Pipeline
The audio signal is processed and transformed into meaningful features:
[ \mathbf{x}{audio} = f{\text{YAMNet}}(s(t)), \quad \mathbf{x}_{audio} \in \mathbb{R}^{1024} ]
- Librosa: Resamples audio to 16 kHz
- YAMNet: Extracts 1024-dimensional embeddings
- Custom DNN: Learns disease-specific acoustic patterns
Prediction from the audio model:
[ \hat{y}{audio} = \text{softmax}(W_a \mathbf{x}{audio} + b_a) ]
3. Clinical Evaluation
A separate branch processes symptom data:
[ \hat{y}{symptom} = \sigma(\mathbf{w}^T \mathbf{x}{symptom} + b) ]
where
[ \sigma(z) = \frac{1}{1 + e^{-z}} ]
- Logistic Regression (LR) estimates baseline disease probability
- Provides essential context that raw audio alone cannot capture
4. Late Fusion Strategy
Both predictions are combined using a weighted approach:
[ \hat{y}{final} = 0.7 \cdot \hat{y}{audio} + 0.3 \cdot \hat{y}_{symptom} ]
- Prioritizes audio intelligence (70%)
- Uses symptoms as a clinical correction layer (30%)
- Thresholding is tuned for maximum sensitivity
5. System Implementation
- Built using Python, TensorFlow, Librosa, and Scikit-learn
- Edge-optimized for real-time inference (< 2 seconds)
- Designed with a guided triage UI ("Health Passport") for usability
Challenges we ran into
- Data Scarcity: High-quality labeled respiratory audio datasets are limited
- Noise Variability: Real-world cough recordings vary in environment and quality
- Multi-Modal Fusion: Combining heterogeneous data (audio + symptoms) required careful weighting
- Deployment Constraints: Ensuring low latency and local processing for privacy and accessibility
Accomplishments that we're proud of
- Built a fully functional multi-modal diagnostic pipeline
- Achieved 90% sensitivity, minimizing missed cases
- Reached 87.5% accuracy on respiratory sound datasets
- Designed a system aligned with WHO standards for community triage
- Delivered real-time results suitable for frontline healthcare use
What we learned
- How to apply pretrained audio models (YAMNet) to healthcare problems
- Importance of multi-modal learning in improving reliability
- Trade-offs between sensitivity vs specificity in medical AI
- Designing systems for real-world deployment, not just lab accuracy
- The role of clinical context in validating AI predictions
AeroTriage aims to become a scalable frontline diagnostic assistant, empowering early detection and improving respiratory healthcare outcomes globally.
