PNEUMOTrack

Track the Signs, Guard the Lungs. PNEUMOTrack is an AI-powered app that screens children's cough sounds on a smartphone to flag early pneumonia risk and guide parents to doctors and nearest hospitals.

Comment

Inspiration

Pneumonia remains one of the biggest health threats to children in Indonesia. In 2018, around 19,000 child deaths in Indonesia were attributed to pneumonia, and in 2023 it was the costliest disease for BPJS Kesehatan, at Rp8.7 trillion. In Surakarta, only 46.4% of estimated pneumonia cases in under-fives were found through health services, so more than half were never reached at all.

The early symptoms look like a common cold, and many parents have no objective way to tell a worrying cough from an ordinary one. X-rays are costly and often unavailable in primary care or remote areas. We asked: what if the phone in every parent's hand could give a first, fast, non-invasive signal?

What it does

PNEUMOTrack is a mobile app for early pneumonia risk screening in children, based on cough sounds.

  1. A parent registers their child and records a short cough with the phone's microphone, guided by on-screen tips (distance, quiet room, 2-3 coughs).
  2. The recording is sent to our backend, which checks audio quality, extracts acoustic features, and runs two anomaly-detection models.
  3. The app shows a clear, non-diagnostic result (normal or needs attention) with a waveform and a plain-language summary.
  4. If risk is flagged, the app recommends next steps: warning signs to watch for, doctors to consult, and nearby hospitals.
  5. Every screening is saved to the child's history so parents can track changes over time.

PNEUMOTrack is a screening aid. It does not diagnose and does not replace a doctor or a chest X-ray.

How we built it

Audio and AI pipeline

  • Preprocessing: mono conversion, RMS energy-based cough segmentation, LogMMSE-style noise reduction (via noisereduce), peak normalization, and 1-second fixed-length chunks (22,050 samples).
  • Features: 20 MFCC coefficients per chunk, summarized with descriptive statistics and mean-pooled into one representation per recording.
  • Because our open dataset contains only pneumonia-labelled coughs (82 patients), we framed the task as one-class anomaly detection. An Autoencoder (reconstruction error against a 95th-percentile threshold) and a One-Class SVM (decision score) each cast a vote, and a majority-voting rule produces the final label.
  • Augmentation: time shifting, pitch shifting and Gaussian noise doubled the training set per fold (82 to 164).
  • Evaluation: Leave-One-Out Cross-Validation (82 folds).

Backend: FastAPI serves the models (.h5 and .pkl), converts the phone's m4a/AAC recordings with ffmpeg, rejects unusable audio (too short, silent, too noisy, no cough detected) with clear error codes, and does not store the audio.

Mobile app: Expo (React Native) with Expo Router, TypeScript and NativeWind, with a calm teal design and large, readable text for parents.

Data and security: Supabase (PostgreSQL) for authentication, child profiles, screening history, doctors, hospitals and articles, protected with Row Level Security so parents only see their own children's data. The backend is containerized with Docker and deployed on a Biznet Gio VPS with HTTPS.

Challenges we ran into

  • One class only. With no healthy-cough data, the model can only say whether a cough resembles the pneumonia pattern, so we had to be careful about what the labels mean and how the app words them.
  • Real phone audio is messy. Android records m4a, not WAV, and home recordings contain TV, fans and speech. We added a quality gate so the app asks for a re-record instead of guessing.
  • Ties between two models. When the Autoencoder and OC-SVM disagree, we surface that uncertainty with an agreement flag rather than hiding it behind a confident number, and treat the tie conservatively as needs attention.
  • Honest UI. Our early designs showed vitals such as heart rate and temperature that cough audio cannot measure. We refused to show invented numbers.
  • Privacy for children. No audio storage, no logging of recordings or child names, and strict per-user data access.

Accomplishments that we're proud of

  • A complete flow from recording to result, history, and referral (doctors, hospitals with distance, and consultation chat), running end to end on an Android build.
  • A dual-model ensemble with a built-in agreement signal.
  • A privacy-first design that respects children's data.

What we learned

  • Anomaly detection is a practical fit when only one class of clinical data is available, but its outputs must be explained carefully.
  • In Leave-One-Out evaluation, 79 of 82 pneumonia samples (96.3%) were classified as inliers and the two models agreed on 78.0% of them. This is consistency with the training class, not clinical accuracy, and it says nothing yet about healthy children or other cough causes.
  • Good screening UX is as much about what the app refuses to claim as what it detects.

What's next for PNEUMOTrack

  • Collect non-pneumonia coughs (healthy children, colds, asthma, bronchitis) to evaluate sensitivity and specificity properly.
  • Run clinical validation with pediatricians and partner puskesmas before any real-world use.
  • Improve robustness to background noise and different phone microphones.
  • Pilot the referral flow (doctors, hospitals, consultation chat) with real health partners, and add anonymized epidemiology dashboards for health offices.
  • Add an iOS version, regional-language support, and regional health-facility data across Indonesia.

Track the Signs, Guard the Lungs.

Built With

Share this project:

Updates

Submission history