Don't Blink, Don't Point

• GIF

  Demonstration of the blinking program detecting blinks

• 

  Some of the code for the Blink detector

• GIF

  Demonstration of the finger direction program

• 

  Some code for the finger direction detector

• 

  Home-made snake. Fresh out the oven

• 

  At the end of the day, we did our very best. But I'm still confused.

What it does

These are implementations of the game Snake, but with a few twists. In the first one (demo_blink), the game speeds up if the user blinks while playing, with the sole intention of making it harder.

In the second one (demo_finger), the player is only allowed to move the snake with their finger by pointing it in a direction parallel to their webcam.

How we built it

OpenCV 4.5, Python 3.8.5. Built the 'game' from scratch.

Challenges we ran into

Multiprocessing was one issue we had at first; Running our programs in parallel required us to use the multiprocessing library, something which we haven't used before. This was a learning experience for everyone involved, and we got past this obstacle successfully.

Finger tracking is far from perfect. While it works fine in some environments, because the process used to isolate and calculate the extended finger and its direction results in a noisy image, oftentimes we encounter interference which results in poor movement quality, and thus, incorrect movements.

Eye blinking is also far from perfect for a similar reason. We essentially have to model the user's eyes and determine if the distance between their eyelids meets a threshold, but this easily false triggers. For example, one of our group mates, who had smaller eyes, continuously triggered the blink detection even when not blinking, to their ire.

Accomplishments that we're proud of

We are really proud that we were able to figure out how to achieve the goals we wanted to within 24 hours. The technology we are working with, OpenCV, is convoluted and usually requires multiple days of effort to figure out even for basic things. Therefore, being able to not only utilize OpenCV to our advantage but also writing two programs that utilize this technology to isolate body parts for alternative game interactions made us extremely proud.

What we learned

We learned that OpenCV's applications are broad; for example, while researching, we saw many others use OpenCV for various things such as facial recognition, image filtering, image resizing, and even motion-based media controls. There are even applications that involve the medical fields, where OpenCV is used to monitor a patient and their vitals. Seeing this versatility made us appreciate the developers of this library more, and gave us an insight into the real-world impact that it has.

What's next for Don't Blink...

Besides the optimization of our recognition algorithms to make them less finicky, we would like to study the real-world impacts that could happen with this software and work on solutions to more issues.

Built With

• opencv-4.5
• python