-
-
Typing the final "n" at the end of "hackpton"
-
Choosing "thank you" among the shortcuts
-
Calibrating
-
I'm tired...
Inspiration
We want to make communication easier for ALS (Amyotrophic Lateral Sclerosis) patients. Since eye movements are usually among the last functions to be affected by this progressive disease, we came up with the idea to use eye tracking to help them communicate with the outside world.
What it does
It allows ALS patients to type using eye movements.
How we built it
We based our code on an open source project (by Antoine Lamé) and openCV to track eyeball movements. We modified it to detect five movements of eyes (left, right, up-left, up-center, up-right), and used non-linear SVM algorithm to calibrate. We then devised a special kind of keyboard to translate eye movements into words.
Challenges we ran into
We spent a good deal of time on figuring out how to do the eye tracking. Another challenge is that none of us knew how to make graphic interface, so that also took some learning.
Accomplishments that we're proud of
We're proud of the whole project. It feels great to see our initial idea slowly developing into a working program. We're especially glad that our effort might be of some help to those people and their family whose life have been negatively affected by ALS.
What we learned
We learned an easy way of eye tracking using webcam. We learned how to make graphic interfaces in Python. And we definitely grew more familiar with Python programming and project developing.
What's next for Unfreeze the Frozen World
There are many possible features to be added. For instance, we can enable the creation of customized keyboard shortcuts so that the program may be more responsive to a patient's specific communication need. We also envision that the program may be connected to home automation systems so that ALS patients may have the ability to do certain things on their own, for instance turning on the light or making a telephone call. In addition, machine learning algorithms (e.g. Google Cloud Text-to-Speech, or AutoML) can help translate the text to audio outputs, and also increase the accuracy of eye tracking.
Log in or sign up for Devpost to join the conversation.