Computers are continually changing and so are the ways we interact with them. Mouse and keyboard are already being replaced by more intuitive interfaces like speech, touch and motion. Technologies leveraging speech recognition algorithms are helping us declare remote controls nostalgic objects of the past. While adjusting the volume or choosing our favorite movies is easier via speech, there are many applications in which physically performing our expected computer behavior is more accurate and intuitive than issuing verbal commands. To name a few, gestures and motion recognition can advance medical applications such as robot-assisted-surgery, improve augmented-reality experience and play a crucial role for computer-aided education.

What We Will Build

We propose Swish: A hand gesture recognition system that can be readily incorporated into various new or existing projects to provide a reliable touchless interface based on hand motion. Swish uses a Leap-Motion sensor device that can be attached to almost any computer as a USB peripheral. On the software side, Swish provides a versatile C++ library that enables any application to declare different hand gestures and register callback functions to be called when their cooresponding gesture is performed by the user. By encapsulating hand gesture classification algorithm as well as the logic for reading and handling motion data, swish aims to provide a library than can easily be integrated into any existing software. The ideal API we would like to develop is described below:

// learning gestures
swish::Learner learner;
learner.learn("gesture-1", std::chrono::minutes(1));
learner.learn("gesture-2", std::chrono::minutes(1));

// classifying gestures
swish::Classifier classifier;
classifier.registerHandler("gesture-1", &func1);
classifier.registerHandler("gesture-2", &func2);;

Come Join Our Team!

If you enjoy problems in machine-learning, particularly support vector machines, and/or development in C++11, this project is a perfect fit for you. Otherwise, you are still more than welcome to join.

Share this project: