What's the Problem?
Presently, touch screens in stores, restaurants, offices and more around the world (otherwise known as kiosks) harbor COVID-19, bacteria and other viruses. By constantly touching these kiosk screens, we increase our chances of catching an infection. This is particularly important at fast-food restaurants like McDonald's, where customers will often eat the food they order at kiosks without washing their hands. Hundreds of others touched that kiosk screen that day, and this only worsens public health.
So what's the solution? (pssttt, it's HOV5R...)
HOV5R is an all-in-one solution for controlling touch screens. By harnessing the power of computer vision (using OpenCV-python), HOV5R allows customers to use simple gesture commands in the air without touching the kiosk to click, scroll, input values and increase or decrease the volume of any sounds being outputted by the machine.
HOV5R works by simply staying in a sleep-like state until a customer holds their hand in a sideways okay hand gesture for a few seconds (the delay ensures the system isn't activated accidentally), which then allows the customer to move their index finger in the air just as if it were the mouse. By bringing their middle finger to their index finger they can click, and pressing the thumb and index finger together will scroll the screen. Then, in order to access volume control of input control all the customer must do is extend all five fingers and swipe left or right, respectively (in the air, of course). And that's it! It's a lightweight solution to a heavy problem, allowing hundreds of customers a day to use simple gestures in the air instead of touching a screen, preventing infectious disease spread and improving public health!
How I built it
HOV5R was built using OpenCV-python with Mediapipe in order to do all the gesture recognition. The frontend of the app was built using Tkinter and the app was packaged using py2exe.