We designed fingerpaint, an app that utilizes visual image processing to implement a drawing tool through motion detection. Using OpenCV, we analyze video streams through a user's webcam footage and use that information to inform the user's interaction with a Canvas. This allows users to more easily whiteboard and prototype basic designs through drawing than with existing systems as Microsoft Paint.
How it Works
When the app first loads, it opens up on an initialization screen that allows users to calibrate the app to their drawing tool of choice. To simplify the model required, we detected a marker on the user's finger using brightly colored electrical tape. After calibration finishes, the user's marked finger is recognized by the app and can be used to interact with the Canvas (implemented with tkinter). Informed by a history of past points, our model detects where the finger is moving and uses this information to move a cursor. When the user presses the 'a' key while the cursor moves, the user can draw on the Canvas.
This was our first time working with OpenCV/Computer Vision in general, so we had a lot of difficulties in determining how best to leverage all the tools the library has and the best way to accomplish the Computer Vision task we had. We had a lot of iteration playing with different approaches to figure out what would work best.
Implement more advanced gesture recognition and UI related features (Undos, Saving Images to Local File, etc.)