Intelli-Vitals: AI-Powered Touchless Health Monitoring

Intelligent Vitals, Immediate Care: Transforming any camera into a clinical-grade health monitor.

Comment

About the Project: Intelli-Vitals 🌟 The Inspiration The primary driver for Intelli-Vitals was the vision of equitable healthcare access. In regions like Bhopal, where specialized medical hardware like ECG machines may not be readily available in every rural clinic or household, we saw an opportunity to turn a ubiquitous tool—the webcam—into a life-saving diagnostic device. We were inspired by the idea that a patient's physical appearance and sub-dermal blood flow could be translated into actionable medical data without a single touch

🛠️ How We Built ItBuilding a clinical-grade tool in a 6-hour sprint required a highly modular and systematic architecture:1. The Perception LayerWe utilized OpenCV and Mediapipe Face Mesh to identify the patient’s forehead and cheeks. This region of interest (ROI) is critical as it has high capillary density and thinner skin, providing the strongest heart rate signal. 2. The Signal Engine (The Math)To ensure the system worked for everyone, we implemented the Plane-Orthogonal-to-Skin (POS) algorithm. This mathematical approach isolates the Blood Volume Pulse (BVP) by projecting normalized RGB signals onto a plane that is orthogonal to the skin-tone direction, effectively neutralizing the "melanin bias" found in simpler rPPG methods. The core projection is defined as:$$P_x = G_n - B_n$$$$P_y = G_n + B_n - 2R_n$$$$S = P_x + \frac{\sigma(P_x)}{\sigma(P_y)} \cdot P_y$$We then applied a 4th-order Butterworth Bandpass Filter to clamp the signal between $0.75 \text{ Hz}$ and $3.0 \text{ Hz}$ (approximately $45 \text{--} 180$ BPM).

  1. The Intelligence LayerWe integrated Google Gemini 1.5 Flash as a "Clinical Lock." While the local Python engine drew the pulse wave, Gemini analyzed 5-second windows of signal data to verify the final BPM and correlate it with uploaded medical reports for a unified diagnosis. ⚠️ Challenges We FacedThe "Transparent Bottle" Bug: Early in development, the algorithm picked up light refraction through a water bottle as a pulse. We solved this by implementing a Face Mesh Confidence Guard that pauses readings if the facial landmarks are unstable or obstructed. Motion Artifacts: Even micro-movements can create noise $1,000\times$ stronger than a heartbeat. We mitigated this by adding real-time stability warnings and detrending the signal. Environmental Lighting: High-frequency flicker from indoor lights often mimicked a pulse. Our bandpass filtering and SNR (Signal-to-Noise Ratio) checks were essential to ensuring only human heart rates were recorded.

📚 What We LearnedThis project was a masterclass in multimodal AI orchestration. We learned that while local signal processing is fast, combining it with an LLM like Gemini provides a layer of "clinical reasoning" that raw algorithms lack. Furthermore, we gained deep insights into the ethics of MedTech, specifically the importance of building inclusive algorithms that perform accurately across all skin tones. By the end of the sprint, we successfully built a platform that not only tracks vitals but provides a direct bridge to Bhopal's healthcare infrastructure—including AIIMS, Hamidia, and Bansal Hospital—via our smart SOS system.

Built With

Share this project:

Updates