Inspiration
We began developing a basic gesture control device to help us navigate about the desktop UI. However while we were messing around with the sensor calibration and movement detection we found that it was a rather simple yet intuitive mouse input. So we decided to design a form of a hands free mouse that would complement the wide range of IO devices available to the differently abled rather than replace them.
What it does
Using a combination of a gyroscope and an accelerometer we were able to determine the pitch, yaw and roll of the hand. With this data we could identify the motion of the hand and hence run scripts accordingly.
How we built it
We used an Arduino with a built in gyroscope and accelerometer to plot the data points, we then calculated the standard deviation to identify movement along any of the three axis. We then print a message to the serial port according to the recognized gesture. Python will then run the script according to the serial values sent to it.
Challenges we ran into
The main challenge that we ran into was identifying what would be classified as a given gesture as the sensor would often fluctuate and gradually vary over time rendering most of our baseline thresholds worthless. We counteracted this by implementing a calibration factor which will help in the normalization of the data.
Accomplishments that we're proud of
We were proud of the fact that this was our very first hackathon and we were able to set our sights on a reasonable goal and through strict time management we were able to complete well ahead of schedule allowing us to optimize the algorithm we used for identifying gestures.
What we learned
We learned how to use libraries specific to the arduino genuini 101 and the how to communicate between the arduino and python, this allowed us to handle data collection using the arduino and plotting and scripting.
What's next for Gesture Control(Hands free mouse*)
We believe the next natural progression for the gesture control would to develop a front end either through a website or an android app which would allow us to calibrate the sensor and configure different gestures to run different scripts. Another major task ahead of us would be to produce a library for the data collection allowing more people to devise more intuitive and capable gesture recognition(eagerly hoping that AI and ML could be used to enhance the gesture control drastically)
Log in or sign up for Devpost to join the conversation.