Inspiration
As a current co-op student at Nuralogix corporation, a machine-learning based health startup, I (Bill) have learned about the use of computer vision and facial blood flow analysis (using deep learning) to predict personal health. Their main product, Anura, involves taking a 30-second video of the user's face, and returning certain health parameters (heart rate, stress index, systolic and diastolic blood pressure, etc.) and disease risks (cardiovascular, heart, stroke, etc.)
Recently, Nuralogix published a paper on the American Heart Association journal, which was picked up by various media sources. Here is the CBC article on the topic.
To showcase the functionalities of my company's product, my teammate and I decided to utilize my company's open-source Python repositories and my company's DeepAffex service to create a web-based application / prototype similar to Anura.
What it does
HealthLens.ai is a web application that resembles the Anura mobile app, which utilizes a computer webcam to record a 30-second video of the user's face, before returning the user's personal health data computed by the DeepAffex engine. It displays the data using dials that are colour-coded for each health category.
How we built it
We decided to use Python Flask since the existing open-source code is in Python, and that we are both well-versed in Python functionalities. In order to receive results, the video needs to be broken into chunks, which are sent to the server using the DeepAffex API. To receive and display results in real time, the Python asynchronous IO was used. This allows sending and receiving concurrently with context switching.
The back-end was based on an open-source end-to-end Python example developed by the company, which includes functionalities such as API registration, video processing, sending and receiving data, decoding, and displaying results.
The front-end was started off with a demo of video streaming in Flask, which was then integrated with our back-end to display the webcam video live on screen, and display the results once measurement is complete. It also provided controls for starting and stopping a measurement. To make the results page, we created a JavaScript component for the dial and scale, which is reused for different health parameters.
Challenges we ran into
Integrating our front-end and back-end involved a lot of structural changes to the existing code. It was also tedious and challenging to work with the Python asynchronous IO, especially when working with video streaming. Since similar programs are usually developed using a JavaScript framework, there isn't a lot of Python documentation / resources to help us, so therefore, it came down to using our Python knowledge and trial-and-error.
What we learned
We significantly underestimated the difficulty of integrating two independent components that were not meant to go together. For any full-stack engineering, it is essential to design the front-end and back-end to be easily compatible and scalable.
What's next for HealthLens.ai
As I (Bill) continue my co-op term with Nuralogix, I am planning to make incremental updates to make HealthLens.ai more mobile-friendly and user-friendly, with overall better UI. I may also include functionalities that differentiate it from the Anura mobile app, such as offering health tips and advice.
Log in or sign up for Devpost to join the conversation.