A member of our team has an uncle who has Parkinson's; while this individual is incredibly intelligent and gifted, unfortunately, their condition leaves it impossible for them to be able to use the computer, as it requires fine motor skills. We built Tacteyele as a solution.
What it does
Using facial detection, Tacteyele watches in which direction your head is moving, and moves the cursor in that direction. In order to click, you wink, and a series of voice commands can be used instead of keyboard shortcuts ("Alt-Tab", "Control-T", etc.), with the ability to dictate speech to instantly be copied/pasted into a text file.
How I built it
Using the Watson API to handle the Speech to Text and SendKeys in Python in order to handle the voice commands, and using OpenCV to manage the mouse.
Challenges I ran into
The OpenCV mouse was very uresponsive/laggy at the beginning of the event. Just as we were about to scrap it, we managed to improve the quality significantly, and ultimately include it in our project.
Accomplishments that I'm proud of
The shear number of voice commands we managed to implement and get working, along with the ultimate success of the OpenCV mouse.
What I learned
Many members of our team needed to learn Python for the first time for our hack, and in researching we learned a lot about UX Design and developing a product like this from beginning to end for a very specific type of user.
What's next for Tacteyele
-More functionality -Different event listeners instead of facial detection and voice for people with different disabilities than Parkinson's