We wanted to tackle a real problem, and based on the experience of a close friend of ours with a heart attack, we started investigating ways to prevent it, or at least to be better informed about our health without intrusive measurement methods.
What it does
Our edge client measures vital signs with computer vision, like beats per minute, respiratory pulse, etc. These are sent in realtime to our DB, and can be managed, as well as calculated alerts if dangerous levels in these signs are present.
How we built it
Computer vision side: Beats per minute, heart rate, and breaths per minute were calculated using Eulerian Magnification of face images in a real-time video. The tool was implemented in OpenCV for simplicity. This client also handles sending the data to the DB in order for it to be displayed in dashboards on the web app.
Web side: Our backend and database are handled in Supabase, our front-end is a static React web app, and we have an additional TS server with hooks listening to real-time updates to the DB to calculate possible alerts, and show them to users.
Challenges we ran into
We skipped the first five hours of the event to attend an ICPC contest. Lighting is always a challenge when dealing with computer vision. Also keeping the subject centered, and only using his face in the video magnification pipeline was important to have accurate results. We tested new web tools but ended up switching to our battle-tested stack (React, Supabase (with PostgreSQL), and Python).
Accomplishments that we're proud of
Making a complete end-to-end system that actually accomplishes the goal. Being able to estimate heart rate with an accuracy of 93% with a webcam.
What we learned
We consulted with a doctor friend of ours before starting the project, to see if this data would be useful to them. They told us that it definitely would be, even in the context of a hospital, where measurement equipment is limited, and can result in long queues which could be time sensitive for patients.
What's next for CamDoc
We want to expand to new edge devices for the measurement of different vital signs, even outside computer vision. Like measuring temperature passively, blood pressure, breathing pulse, and face deformations (which is an important and predictive sign for brain strokes).
The good thing is that we designed the web interface to be general enough to support these different vital signs, and even generated some sample data to test it during our demo.
Log in or sign up for Devpost to join the conversation.