Blinkr

Inspiration

A human eye on average blinks up to 20 times a minute. But, this rate drops to less than half while we look at our computer and smartphone screens. This reduced rate of blinking leads to consequences like vision changes and dry eyes. Blinkr was built to combat this and help the users maintain a good eye health.

What it does

Blinkr lets the user set a time interval (10 seconds by default) before starting the session. Once the session begins, the app starts checking for blinks in the time interval specified. If the user has not blinked in the given time interval, the app sends a beep alerting the user to blink.The app is built to ensure that the user blinks at least once in the set time range. Note:- Blinkr currently works well with reading activities on the computer. The app would soon be made compatible for typing activities in the future versions.

How it is built

Blinkr is made on top of python and utilizes mediapipe framework for face detection and landmarking, and OpenCV for capturing video.

• It all begins with OpenCV capturing video through the webcam which is then processed by face-mesh which detects the face in the video frame and provides 468 landmarks on the face.

• The frame given by OpenCV is in BGR by default but mediapipe processes RGB data.So, the BGR frame is converted to RGB before letting mediapipe process it.

• Out of the 468 landmarks given we take just the landmarks that surround the right and left eye.

• We then find the vertical and horizontal distance between the extremes of the eye(horizontal and vertical) and divide them(vertical distance/horizontal distance) to get the eye aspect ratio (EAR) and start counting blinks (It is said to be a blink when the EAR drops below a threshold (set as 0.3 in our app)).

• A thread is run parallelly which executes a function that checks if the blink count has been updated within the time duration set, and produces a beep if the blink count has not been updated.

Challenges we ran into

• An optimal EAR threshold value was to be obtained after running trials on multiple human subjects as user behavior is not consistent.
• It was also observed that when the user looks down at the keyboard while typing, the program mistakes it for a blink and might deviate from the required behavior. To solve this, a calibration can be done before the session begins and set exceptions for those particular behaviours of the subject.

What we learned

This was a great opportunity to explore the capabilities of face recognition and detection and their applications. It was while building this project, I came to know about the working of social media filters that use the very same solution of landmarking on a face mesh.

What's next for Blinkr

• The future versions of Blinkr will support typing activities on laptops and desktop computers by introducing a calibration phase before beginning a session.
• Blinkr will also be made available on the browser in the form of a website and an extension so that an installation is no more required.
• A system that rewards the user for having a healthy blink rate during the session shall also be introduced to make maintaining eye health more playful.

Built With

• mediapipe
• python
• tkinter