EyeAlert

Inspiration

Those in prolonged, attention-demanding situations may find that fatigue can be a hinderance towards successfully completing their tasks.

What it does

The machine learning (ML) model analyzes eye movements to alert users when their attention deviates from the task at hand. The React Native app shows the user's history of fatigue and can help them identify trends in their concentration (or lack thereof) over time in demanding environments. This is a preventative measure to decrease fatigue-based accidents in critical situations.

How we built it

We built a solution that uses React Native and a machine learning model to track eye movements and display trends of fatigue during critical situations.

Key components:

Machine Learning model to track eye movements of a target user in a continuous live feed
State changes in blinking movements are recorded and relayed to alert software
Individuals are alerted if target user’s blinking movements indicate fatigue
User can securely access personal profile afterwards on an interface to visualize trends on when they were fatigued and enables them to find ways to minimize fatigue

Challenges we ran into

Spent a lot of time on the ideation phase
Limited knowledge as a team in the application domains of our project in order to contribute an unique technical solution
Learning and integrating disparate technologies from each part of the solution

Current limitations

ML model to detect eye gaze movements is not perfectly accurate
Implicit bias in algorithm to detect when an user’s eyes are closed because of diversity in eye shapes and size

Accomplishments that we're proud of

Entire project is demo-ready
Broad set of application domains

Next steps

Improve ML model to detect eye gaze movements with a large data set and high number of iterations Current implicit bias in the algorithm to detect when an user’s eyes are closed can be improved by using a more diverse data set
Focus on using wearable technology
Further expand application of solution to apply to defense, law enforcement, medical situations, distracted driving, etc.

Value proposition - Children's Health

4 pillars of this solution.

Personalization: key is an individualized fatigue profile (IFP)
- empowers residents and surgical professionals to improve themselves through gentle positive feedback (since it's already a high-pressure environment)
  - alleviate pressure, not create it. positive reinforcement
- Empowers them to understand when they can develop better fatigue management
Training: if they are better equipped to handle these stressful situations, they can be empowered to advance faster via analysis of the IFP
Optimization: Aggregation at the provider level can optimize scheduling to ensure that residents are doing well
- Benefits providers because less fatigue == fewer mistakes, which costs them less money
Building for the future: We also would like to integrate with emerging technologies

Value proposition - Lockheed Martin

Two key pillars.

Edge computing:
- To simulate running on an edge computing device, we ran the machine learning (ML) model on a limited-spec laptop without a GPU
- Analyses of eye movements are run offline to preserve confidentiality in the interest of national security and to reduce reliance on a stable internet connection
Secure cloud computing:
- In a defense-focused deployment of EyeAlert, we would focus on security by deploying to private clouds like AWS GovCloud or Azure for US Government
- Fatigue profile iOS app would NOT be publicly available, would be deployed on to gov-owned mobile devices privately