As someone who often takes notes from a laptop screen, it takes a few seconds to stop writing with a pen to scroll with the mouse or trackpad. With In Your Face, I can free up both hands to take notes. This only saves seconds, but those will add up, leading you to wonder where that extra hour came from at the end of the week.

What it does

In Your Face uses facial feature tracking to allow for gesture control. Specifically, blinking causes a short downwards scroll while opening your mouth causes a short upwards scroll. I chose to do it this way because blinking is a natural action that happens while you're reading, which minimizes disruption to your focus, while opening your mouth is an easy way to correct the program if you blink too many times. Looking away from the screen also stops the computer from detecting your face, which is great if you have to look down to write notes. Furthermore, the window can be minimized and the application will still function, allowing for seamless integration into a normal work environment.

How I built it

I used Python because I have previous experience in facial recognition APIs. This time, I used OpenCV for Python and dlib to detect blinks. I modified another user's blink detection program to suit my needs and learn a bit about OpenCV and dlib at the same time, then I wrote my own method to detect the mouth opening.

Challenges I ran into

DeltaHacks VI is the first time I've flown solo at a hackathon. This is a very different experience than working in any group because you don't have any other people to bounce ideas off of. Because of this alone, it took me a long time to come up with the idea for In Your Face. I made friends in other groups to share my ideas and learned a bit about their projects in the meantime.

Accomplishments that I'm proud of

I'm proud of how I thought of my idea and made it happen. I'm usually not the most creative person, so to have a good idea in itself is a huge accomplishment to me.

What I learned

Even though I usually prefer to work alone, I learned the real value of a team. I already knew that working with a team lets you divide up the work and have multiple different areas of expertise, but I never imagined just how hard solo development is.

What's next for In Your Face

I can see myself keeping In Your Face for personal use, whether it be for its intended note-taking purpose, or just for lazily scrolling through social media. In the future, In Your Face could be made better by allowing the user to configure the thresholds for the gestures, which could allow each person to calibrate the program to suit their own face.

Built With

Share this project: