Ender's Keyboard uses computer vision to detect the user's hand in midair, allowing the user to type by dragging several virtual keys. It also includes shortcuts to various websites such as google, youtube, or wikipedia. The virtual keys will follow the user's hand as it moves. The keyboard is capable of typing into any application and is meant to provide extra convenience, such as typing from a distance or with grimy hands, and prevent the spread of germs on public devices.

The Story

Ender's Keyboard is inspired by a one-handed touchscreen keyboard using a similar drag-to-type function that was shown in the movie "Ender's Game."

The Team

We are a two-person team who met at HSHacks. Derek is an avid programmer and is on a local, entirely student-led robotics team. He has been coding for 5 years and has a great deal of experience with artificial intelligence. Derek previously mentored at Superposition II, an all-girls hackathon in the bay area and participated in ExpoHacks, a 12-hour hackathon, also in the bay area. David has never been to a hackathon before, but greatly enjoyed HSHacks and is happy he was able to create such a great product at his first one.

The Journey

Derek began working on the computer vision while David figured out how to search the web and simulate keyboard input. Having not used python for a major project in a while, Derek had to take some time to familiarize himself with the language. He had used OpenCV for computer vision in robotics, however, so he was familiar with the library and encountered only minor difficulty translating his Java/C++ experience to the python bindings for OpenCV. He was also slightly unused to the lack of real-time error checking and the strange format of OpenCV's and numpy's arrays. However, two of the largest frustrations were trying to get convexity defect detection, which detects concave "valleys" in shapes such as hands, to work. Though he found the convexity defects he needed, it also detected many minor defects. Filtering by size seemed to remove large defects just as much as tiny ones. The second frustration was figuring out how to type only a single letter per key stroke. At this point Derek was very tired and took a long time to implement a simple toggle switch, which solved the problem. Meanwhile, David had trouble installing many of the necessary libraries as he was on a Windows computer and had conflicting versions of python.

Built With

Share this project: