Using computer vision to infer medical anomalies from eye data.




This project was inspired by HGN tests for DUIs. We wanted to use computer vision to detect anomalous eye movement like jitteriness, haziness, and general nystagmus. We used image segmentation techniques to track small eye movements that are not easily detectable by humans. Hopefully, in the case of DUIs, this can help eliminate bias and error. Furthermore, we realized that the data our program captures is incredibly powerful -- our simple and accessible GUI , can be used by first responders, medical practitioners, and even the "average Joe" in detecting underlying conditions like fatigue and intoxication.

Development Process + Difficulties

Initially, we came into TreeHacks with the idea of quantifying and analyzing eye movements accurately with a simple laptop, low-quality camera. Our main motivation was providing an easily accessible, free, data-driven diagnostic tool for everybody.

We first attempted to tackle this problem by applying classic algorithms and tools like using the Haar Cascade classifier, Hough Circle Transform, the Canny Edge Detection algorithm, and binary threshold operations. Unfortunately, none of them could isolate the pupil quite accurately enough for us to track using our embedded laptop webcams.

Although initially promising, many of the techniques we attempted to implement were not suited for poor webcam video quality, non-ideal ambient lighting, and dark iris colors.

As a result, we created our own algorithm that leverages the "shape_predictor_68_face_landmarks.dat" model. In order to combat dim lighting, we applied a filter to optimize the brightness, contrast, and exposure of input images.

Future Steps

We plan to quantify and analyze pupil dilation and irregularities which are crucial indicators of concussions and other nervous system injuries. We also hope to improve tracking by constructing our own Masked CNN which would reduce statistical noise.

How to install:

To run this file, install all dependencies

git clone
cd VerifEye
virtualenv venv
pip install -r requirements.txt

Created by Sunny Cui, Anthony Bao, and Ethan Chiu

Built With

Share this project: