One of our team members wanted to make an alternative mouse input to play osu!, a rhythm game, without a mouse. We figured the webcam would be a fun place to start!
Manouse is a portmanteau of "manus", the Latin word for hand, and "mouse."
What it does
Manouse uses computer vision technology to map hand gestures to mouse control. We use OpenCV to create an outline of the hand, find it's position, and count the number of fingers. The position and number of fingers data can be mapped to various actions. Currently the position moves the mouse and finger count toggles clicking.
How we built it
We used OpenCV, win32api (to move the mouse), and Python
Challenges we ran into
We had to filter out faulty readings of the number of fingers, and develop robust algorithms and functionalities to compensate for inaccuracies.
Accomplishments that we're proud of
We are all first year Cal students, with the majority of us having little to no programming experience before this Fall.
What we learned
We learned how to use OpenCV to process frames captured by a computer's webcam, and using powerful algorithms, resolve meaningful information from the frames. We also learned how to use Python's win32api to control mouse position, mouse movement, and other core PC functionalities.
What's next for Manouse -- Webcam mouse
We look forward to expanding it's functionality, making a more friendly user interface, and improving it's hand detection and background filtering solutions.