-
Dashboard overview showing the live Virtual rPPG Pipeline interface, current estimate status, and session controls.
-
ROI quality map identifying the facial regions used to extract color signals for heart-rate estimation.
-
Heart-rate trend view showing validated rPPG estimates over time and recent backend signal-status events.
-
Signal-quality dashboard explaining whether the current pulse estimate is accepted or withheld by the backend.
-
Trial logging panel for comparing the webcam-based rPPG estimate against a reference heart-rate measurement.
-
Live scan modal showing webcam face tracking, ROI overlays, BPM estimate, confidence, and real-time quality metrics.
Inspiration
Virtual appointments are becoming more common, but they still lack some of the quick physical context doctors get in person. I wanted to explore whether a normal webcam could provide a lightweight, contactless heart-rate estimate using computer vision, without requiring a wearable or external sensor.
What it does
Virtual rPPG Pipeline estimates heart rate from webcam video by analyzing subtle color changes in facial skin. The app detects facial regions, extracts RGB signals, filters them into a pulse-like waveform, estimates BPM, and shows signal-quality metrics so users can see when the reading is reliable. It also includes a scan diagnostic mode and trial logging so estimates can be compared against a reference BPM.
How I built it
I built the backend in Python using Flask, OpenCV, MediaPipe, NumPy, and SciPy. MediaPipe detects facial landmarks, OpenCV processes webcam frames, and the rPPG pipeline extracts color signals from cheek and forehead regions. I implemented POS/CHROM-style signal processing, bandpass filtering, BPM estimation, confidence scoring, and a browser-based dashboard using HTML, CSS, and JavaScript.
Challenges I ran into
The biggest challenge was making the readings transparent and defensible instead of simply outputting a number. rPPG is very sensitive to lighting, motion, camera quality, and frame-rate instability, so I added signal-quality gates, scan diagnostics, and trial results to show when the system is confident versus when it is withholding a reading. I also added a comparison layer where users can enter a reference heart rate from a smartwatch, pulse oximeter, or manual pulse count, allowing the app to calculate error between the pipeline estimate and a more established measurement.
Accomplishments that I'm proud of
I’m proud that the project includes a full pipeline from face detection to signal extraction, filtering, BPM estimation, quality scoring, diagnostics, and trial comparison. I’m also proud of making the interface transparent, so users can inspect the signal quality and understand why a reading is or is not being shown.
What I learned
I learned how remote photoplethysmography works, especially how small color changes in facial skin can be transformed into a heart-rate estimate. I also learned that computer vision health tools need strong uncertainty handling, because showing no result can be more responsible than showing a misleading result.
What's next for Virtual rPPG Pipeline
Next, I would improve accuracy by testing against public rPPG datasets and real reference devices, adding better motion compensation, improving skin-region selection, and collecting more structured validation trials. I’d also like to add exportable reports and support for additional signals like respiration rate if the video quality is high enough. Longer term, I would explore multimodal validation by combining webcam-based rPPG with other low-cost sensors or reference devices to make contactless vital-sign estimation more reliable.
Log in or sign up for Devpost to join the conversation.