EyeLS: Non-Verbal Communication via Gaze Tracking

Inspiration

In 2017, my uncle was diagnosed with amyotrophic lateral sclerosis (ALS), just one of many neurodegenerative disorders that affect more than 50 million people worldwide. The gradual death of nerve cells associated with conditions like ALS establishes neurodegenerative disorders as the leading cause of physical disability worldwide. The progressive degeneration of neurons in the central and peripheral nervous systems presents significant mobility and speech impairments for individuals affected by these disorders.

Effective treatments for several neurodegenerative conditions remain elusive, and countless patients struggle to navigate daily life without the ability to communicate verbally. Although there have been attempts to develop technology such as augmentative and alternative communication (AAC) systems or speech-generating devices (SGDs), significant challenges persist due to the inadequate availability and affordability of these devices. Recognizing the inherent need for a non-verbal communication device, I developed EyeLS: a novel, accessible gaze-tracking application for patients with neurodegenerative disorders.

What it does

Despite significant motor impairments, ocular control functions and gaze fixation remain intact for most patients of neurodegenerative disorders. Therefore, leveraging gaze-tracking technology is an optimistic solution to provide non-invasive, accessible communication for patients with neurological inflictions.

EyeLS infers the eye-gaze location of users in real-time using common webcams that are already present in most laptops, smartphones, and mobile devices without the need to purchase additional equipment. The application uses webcam feature detection to calibrate eye movements and associate that with the location of users' eye fixture. After calibration, the application presents an eye-tracking keyboard and a text-to-speech option to communicate non-verbally.

How I built it

EyeLS utilizes ridge regression, a machine learning algorithm that analyzes the multicollinearity in data and shrinks regression coefficients using L2 regularization. The algorithm implements weights and a penalizing term to overcome overfitting bias when generating a regression model to predict gaze locations on the screen.

Alternative eye-tracking technology in the status quo has been hindered due to saccadic movement and imprecise calibration, but EyeLS overcomes these limitations through its implementation of mathematical models that enhance accuracy and reduce interference. In particular, the ridge regression model utilizes Kalman filtering, a linear quadratic estimation (LQE) algorithm that produces estimations by observing statistical noise and inaccuracies over time.

The synergy between EyeLS’s ridge regression model and layer of Kalman filtering enables the system to learn and anticipate saccadic movements, resulting in a smoother and more natural eye-tracking experience. While gaze tracking is invariable and sporadic without mathematical recursivity, the Kalman filter optimizes both user experience and data processing in the front-end and back-end environments.

Challenges I ran into

Initially, I faced challenges in optimizing the accuracy and precision of EyeLS. By simply implementing ridge regression, eye movements were sporadic and uneven. In addition, calibration was not effective and was not able to account for retinal disparities between both eyes. However, I overcame this challenge by implementing Kalman filtering, which significantly improved the stability and smoothness of eye movement tracking. Although understanding the math behind this model was perplexing at first, I invested time in studying the underlying principles and mechanics of Kalman filtering. This deep dive into its mathematical foundations enabled me to fine-tune the filter parameters effectively. As a result, I was able to achieve more accurate and reliable predictions of eye positions.

Accomplishments that I'm proud of

One of the most significant accomplishments with EyeLS is its ability to deliver high-precision gaze tracking using only standard webcams, making it an accessible and affordable solution for users. By integrating advanced machine learning algorithms and mathematical models, EyeLS achieves a level of accuracy and stability that is comparable to more expensive, specialized eye-tracking hardware. This democratization of technology ensures that individuals with neurodegenerative disorders can access effective communication tools without financial barriers. Knowing that this tool has the potential to make a meaningful impact on the daily lives of patients and their caregivers is incredibly fulfilling.

What I learned

Delving into machine learning algorithms, particularly ridge regression, and understanding the intricacies of Kalman filtering, required a strong foundation in both computer science and applied mathematics. This project emphasized the need for continuous learning and adapting new concepts to overcome technical challenges.

I learned that addressing real-world problems involves not only technical proficiency but also empathy and understanding of the user's experience. User-centric design emerged as a crucial aspect of the development process. Engaging with potential users and incorporating their feedback was instrumental in refining the application's usability and effectiveness.

What's next for EyeLS

In future research, it would be valuable to explore the scalability and generalizability of EyeLS across diverse populations and neurological conditions. My objective for future research is to ensure that EyeLS does not discriminate against racial/ethnic groups, age demographics, or other potential factors that could introduce bias.

Additionally, I seek to incorporate a customizable interface that allows users to store certain words/terms, so they do not have to type it out each time. For example, there could possibly be a button that relays the message "I need to use the restroom" or "I am hungry," eliminating the need to spell out commonly used phrases repeatedly. This feature would further enhance the efficiency and user-friendliness of EyeLS, making it even more practical and supportive for daily communication needs.