Inspiration
I am a heart disease survivor who has went through open-heart surgery twice and undergoes heart evaluations such as electrocardiograms(ECG) often to ensure good health. I wanted to build something for patients like I that would enable affordable, frequent access to evaluations they need, when they need it.
Having to schedule potentially weeks ahead for in-person ECG appointments seems inefficient, where the same data can be achieved at home in 30 seconds using photoplethysmography(PPG) techniques. In short, it uses the brightness of a fingertip that changes with pulse to calculate ECG-like data.
I became inspired to apply this concept to a mobile app for accessibility and everyday use.
What it does
The mobile app portion of this project allows patients to perform a PPG based scan on their fingertip, recording the changing brightness values to calculate their pulse, as well as forwarding the scan in ECG data format to doctors. Github link here.
The web app portion of this project allows doctors to create and manage patient accounts, and view their patient's data which they have uploaded to the cloud. From here doctor's can perform certain evaluations on particular scans, such as arrhythmia detection, QT interval measurement, and ST segment analysis. Github link here.
As of now, a machine-learning model exists to identify arrhythmias in a normalized dataset. It will indicate if arrhythmias exist and alert the doctor.
How we built it
The mobile app is built using the Flutter framework to support both Android and iOS devices. All the code written by me in the app is in Dart, and I utilize the Flutterfire package to authenticate users with my firebase project.
The web app is built using the Next.js framework, using React and Tailwind CSS. The code written in this project is primarily Typescript, and I also handle database and user authentication using Firebase. The model is accessed via a web API endpoint developed in the Next.js website. REST architecture used.
Challenges we ran into
Normalizing data to fit the input of our machine-learning model. To scale it from brightness value format to model readable ECG format, I thought it would be a good idea to use min-max scaling techniques since I wanted to preserve the original distribution of the data, as this represents the pulse. Thus I scaled the values down to values between [0,1], normalized in the same manner as the model.
Developing an accurate algorithm to calculate the pulse of the user from brightness values. To overcome this challenge, I identified peaks by checking the 2 values before and after a certain point, to see if it was a just a small fluctuation in the camera lighting or a significant change in lighting due to the actual finger pulse.
Accomplishments that we're proud of
Bridging a mobile app to a website via cloud platform
Creating a high accuracy machine-learning model, and figuring out how pre-processing/scaling things to match a common range of values really work.
Attempting to build an accessible health care solution that could help prevent serious issues
What we learned
- Different techniques to use when handling large datasets and the common classifiers used
- How to structure user hierarchy in a database
- Good documentation practices
- Testing on iOS devices from a Windows system is very difficult
What's next for HeartLens
- Enhancing the pulse calculations
- Identifying flawed scans where finger tip may not be completely on phone screen
- Enabling other evaluations beyond arrhythmias to be performed by doctors