We wanted to learn OpenCV and convolutional neural networks. This was the first idea that came to us which could use both technologies.

What it does

The Android application utilizes the front facing camera and detects the users blink rate. Research has shown the blink rate is correlated to drowsiness. This application would be used in a car to alert a driver when they reach a certain blink rate threshold to keep them from falling asleep at the wheel.

How we built it

The original plan was to use OpenCV to perform facial recognition and extract an image of a face out of the Android phones front facing camera. We were then going to pipe that image to a trained TensorFlow CNN to identify if the user's eyes were open or closed. It took a long time just to get OpenCV running on our Android app so we ditched the TensorFlow component and decided to rely on OpenCV entirely. The way the application works is it uses open CV to extract the portion of the image where the users eyes are. The image that is extracted is turned to greyscale and the sum grayscale intensity of every pixel is taken. We then put this into a small buffer of previous data points. We compute the standard deviation of this small buffer against a larger long running average of the standard deviation. When we see large swings from the "normal" variance we trigger a blink detection.

Challenges we ran into

Originally we noticed that the pupil location moved a little when you blinked your eyes. This was a side effect of OpenCV's eye detection algorithm. We used the position of the pupil to derive it's acceleration from previous points and attempted to trigger off of this acceleration. We put a lot of effort in trying to calibrate this but in EE terms the signal was below the noise floor. There were as many false positives as true detections and we could not get an accurate system.

Accomplishments that we're proud of

We're mostly proud of learning a lot about the OpenCV platform and to be honest having a cool looking app. We display all of the recognition boxes on the screen and it just looks cool :D

What we learned

We learned a lot about OpenCV mostly and certainly the challenges which present themselves even after you seemingly have "the hard part" done.

What's next for Daydream Detector (AKA Anti-sleep 5000)

Daydream Detector needs a more polished UI and we can still make improvements on accuracy. There are a few false positives and it doesn't work perfectly on everyone's face.

Built With

Share this project: