Inspiration
Really a silly source of inspiration, but more or less I was inspired to make this after the left click buttons on my mouse broke, and I was left to deal with a trackpad for over a week. This led to irreversible (negligible) pain both physically and emotionally; therefore, I sought to replace the travesty that is the trackpad with hopefully, at least in my eyes, something more convenient.
What it does
Handy takes your generic webcam and turns it into a mouse sensor - you can now use your hand as if you had a dollar store mouse duct taped to it! With Handy, you can achieve all basic functionality, including clicking, scrolling, dragging, and obviously moving. We've also designed the controls to be as intuitive as possible to make it as user-friendly as possible.
How we built it
We built Handy with python - we used opencv to get camera data and used mediapipe to locate where each point of interest on the hand was. We also used PyQt5 for the GUI and interface.
Challenges we ran into
One of the main challenges was getting the cursor to not feel as jittery; because your hands wouldn't be exactly still midair, we would have to implement a moving average filter to reduce input noise while still remaining a sharp response.
Accomplishments that we're proud of
First time for everything, but we're still proud of just how far we've come in our first hackathon. There was a lot of effort put into this over the weekend, but we're just happy with how it came out.
What we learned
Don't overcomplicate things - we initially tried to go way bigger with an onscreen keyboard but quickly realized that it was basically impossible, so we scaled back. Scaling back helped to make the project a lot more lightweight, more like a tool, not a fully fledge desktop suite.
What's next for Handy
Potentially move the tracking off of a bad laptop webcam and develop some real hardware. This would greatly benefit the hand recognition and also help to make the inputs appear more responsive.
Log in or sign up for Devpost to join the conversation.