Cough AI

• 

  Cough AI - Home Page

Inspiration

We were inspired by the idea that something as simple as a cough or a facial expression could reveal early signs of illness. With healthcare becoming increasingly digital, we wanted to build a non-invasive, accessible tool that gives people quick, AI-powered health feedback using just their devices.

What it does

Cough AI is a web-based application that analyzes a user’s cough and facial cues to provide personalized health insights. It classifies coughs (e.g., bronchitis, whooping cough, or normal) using a model trained on the Kaggle Cough Audio Dataset. At the same time, it performs real-time facial analysis to assess fatigue or distress, combining both to generate a natural-language health recommendation using the Gemini API.

How we built it

We built the frontend using React and Tailwind CSS for a clean, responsive interface. For audio processing, we used the Web Audio API to record or upload cough samples and extract MFCC features for classification with a TensorFlow.js model. We used MediaPipe and WebRTC for real-time facial landmark detection to estimate fatigue or stress levels. The final diagnosis is generated through the Gemini API, and the app is deployed via Vercel for easy access.

Challenges we ran into

• Handling microphone and camera permissions reliably across browsers and devices
• Ensuring consistent audio classification with varied cough quality and recording environments
• Integrating facial recognition with limited training data and in-browser processing
• Processing audio and video in real time while maintaining performance and user experience

Accomplishments that we're proud of

• Creating an intuitive user interface that simplifies a complex AI pipeline
• Using the Gemini API to output natural, readable health recommendations

What we learned

We learned how to implement real-time audio and video processing in the browser, how to train and deploy lightweight machine learning models with TensorFlow.js, and how to make AI more approachable through good UX and language. Most importantly, we saw the potential for combining multiple data streams to improve health diagnostics.

What's next for Cough AI

Next, we plan to improve the accuracy of our models by fine-tuning them with more diverse datasets and expanding our condition labels. We’d also like to support mobile devices via React Native, introduce additional health cues like breathing patterns or heart rate, and explore integration with telemedicine platforms for remote pre-screening.

Built With

• and-the-gemini-api
• api
• css
• datasets
• devtools
• gemini
• html
• javascript
• kaggle
• mediapipe
• react
• tailwind
• tailwind-css
• tensor
• tensorflow
• tensorflow.js
• vercel
• web
• web-audio-api
• webrtc