Inspiration
In 2017, my uncle was diagnosed with amyotrophic lateral sclerosis (ALS), just one of many neurodegenerative disorders that affect more than 50 million people worldwide. The gradual death of nerve cells associated with conditions like ALS establishes neurodegenerative disorders as the leading cause of physical disability worldwide. The progressive degeneration of neurons in the central and peripheral nervous systems presents significant mobility and speech impairments for individuals affected by these disorders.
Effective treatments for several neurodegenerative conditions remain elusive, and countless patients struggle to navigate daily life without the ability to communicate verbally. Although there have been attempts to develop technology such as augmentative and alternative communication (AAC) systems or speech-generating devices (SGDs), significant challenges persist due to the inadequate availability and affordability of these devices. Recognizing the inherent need for a non-verbal communication device, I developed EyeLS: a novel, accessible gaze-tracking application for patients with neurodegenerative disorders.
What it does
Despite significant motor impairments, ocular control functions and gaze fixation remain intact for most patients with neurodegenerative disorders. Leveraging gaze-tracking technology, therefore, offers an optimistic solution to provide non-invasive, accessible communication for these patients.
EyeLS infers the eye-gaze location of users in real-time using common webcams found in most laptops, smartphones, and mobile devices, eliminating the need for additional equipment. Implemented in Python, the application utilizes webcam feature detection to calibrate eye movements and associate them with the location of the users' eye fixation. After calibration, the application presents an eye-tracking keyboard and a text-to-speech option, enabling non-verbal communication. Python's robust libraries and tools, such as OpenCV for image processing and scikit-learn for machine learning, were instrumental in developing and fine-tuning these features.
How I built it
EyeLS utilizes ridge regression, a machine learning algorithm that analyzes multicollinearity in data and shrinks regression coefficients using L2 regularization, implemented in Python. By leveraging Python's libraries such as NumPy, SciPy, and scikit-learn, the algorithm assigns weights and a penalizing term to overcome overfitting bias when generating a regression model to predict gaze locations on the screen.
Alternative eye-tracking technology in the status quo has been hindered due to saccadic movement and imprecise calibration, but EyeLS overcomes these limitations through its implementation of mathematical models coded in Python that enhance accuracy and reduce interference. Specifically, the ridge regression model utilizes Kalman filtering, a linear quadratic estimation (LQE) algorithm implemented using Python libraries such as filterpy. This algorithm produces estimations by observing statistical noise and inaccuracies over time.
Challenges I ran into
Initially, I faced challenges in optimizing the accuracy and precision of EyeLS. By simply implementing ridge regression using Python, eye movements were sporadic and uneven. In addition, calibration was not effective and could not account for retinal disparities between both eyes. However, I overcame this challenge by implementing Kalman filtering using Python libraries, which significantly improved the stability and smoothness of eye movement tracking.
Understanding the math behind this model was perplexing at first, but I invested time in studying the underlying principles and mechanics of Kalman filtering. Utilizing Python's comprehensive ecosystem, including libraries such as NumPy and filterpy, I was able to dive deep into the mathematical foundations and effectively fine-tune the filter parameters. As a result, I achieved more accurate and reliable predictions of eye positions.
Accomplishments that I'm proud of
One of the most significant accomplishments with EyeLS is its ability to deliver high-precision gaze tracking using only standard webcams, achieved through Python programming. By integrating advanced machine learning algorithms and mathematical models using Python libraries such as scikit-learn, NumPy, and filterpy, EyeLS attains a level of accuracy and stability comparable to more expensive, specialized eye-tracking hardware. This democratization of technology, facilitated by Python's powerful and accessible tools, ensures that individuals with neurodegenerative disorders can access effective communication tools without financial barriers. Knowing that this tool, developed extensively using Python, has the potential to make a meaningful impact on the daily lives of patients and their caregivers is incredibly fulfilling.
What I learned
Delving into machine learning algorithms, particularly ridge regression, and understanding the intricacies of Kalman filtering, required a strong foundation in both computer science and applied mathematics, which I leveraged using Python. This project emphasized the need for continuous learning and adapting new concepts to overcome technical challenges, with Python serving as a versatile tool to implement and experiment with these advanced algorithms.
I learned that addressing real-world problems involves not only technical proficiency but also empathy and understanding of the user's experience. User-centric design emerged as a crucial aspect of the development process. Using Python, I was able to quickly prototype and refine the application, incorporating feedback from potential users to enhance usability and effectiveness. Engaging with users and utilizing Python's rich ecosystem of libraries and tools was instrumental in refining EyeLS and ensuring it met the needs of its users.
What's next for EyeLS
In future research, it would be valuable to explore the scalability and generalizability of EyeLS across diverse populations and neurological conditions. My objective for future research is to ensure that EyeLS does not discriminate against racial/ethnic groups, age demographics, or other potential factors that could introduce bias.
Additionally, I seek to incorporate a customizable interface that allows users to store certain words/terms, so they do not have to type it out each time. For example, there could possibly be a button that relays the message "I need to use the restroom" or "I am hungry," eliminating the need to spell out commonly used phrases repeatedly. This feature would further enhance the efficiency and user-friendliness of EyeLS, making it even more practical and supportive for daily communication needs.
Log in or sign up for Devpost to join the conversation.