With Family members being at different locations and in certain health situations, on video calls it is reassuring to know each others pulse. This can also be applied in Telehealth settings and there is no privacy or data issue as the face detection and pulse is calculated on the mobile device itself!
What it does
The pulse algorithm is on-device and no personal data is sent to the cloud. First, face detection is done and based on the pixel level 'RGB value' changes, pulse ( + or - 5 ) is computed.
How I built it
This was built using Agora video SDK for Android and a custom pulse algorithm (based on MIT research) Agora Pulse is licensed under 'MIT License'
Challenges I ran into
Ideal lighting conditions (without a bright light source in the camera viewfinder) and the person holding still is advisable
Accomplishments that I'm proud of
Applying the pulse detection on the remote user video stream
What I learned
Setting up the video stream based on the phone resolution Integrating advanced video API for call enhancement
What's next for Agora Pulse
Adding users via Agora RTM SDK Fine tune the custom pulse algorithm to reduce the variance Display info on ideal lighting state or not