EyeWare

An innovative eye-tracking solution leveraging machine learning algorithms and the AdHawk MindFlex pupil tracking glasses to enhance driver attentiveness.

Comment

Inspiration

In 2019, close to 400 people in Canada died due to distracted drivers. They also cause an estimated $102.3 million worth of damages. While some modern cars have pedestrian collision avoidance systems, most drivers on the road lack the technology to audibly cue the driver if they may be distracted. We want to improve driver attentiveness and safety in high pedestrian areas such as crosswalks, parking lots, and residential areas. Our solution is more economical and easier to implement than installing a full suite of sensors and software onto an existing car, which makes it perfect for users who want peace of mind for themselves and their loved ones.

What it does

EyeWare combines the eye-tracking capabilities of the AdHawk MindLink with the onboard camera to provide pedestrian tracking and driver attentiveness feedback through audio cues. We label the pedestrian based on their risk of collision; GREEN for seen pedestrians, YELLOW for unseen far pedestrians, and RED for unseen near pedestrians. A warning chime is played when the pedestrian gets too close to get the driver.

How we built it

Using the onboard camera of the AdHawk MindLink, we pass video to the openCV machine learning algorithm to classify and detect people (pedestrians) based on their perceived distance from the user and assign them unique IDs. Simultaneously, we use the AdHawk Python API to track the user's gaze and cross-reference the detected pedestrians to ensure that the driver has seen the pedestrian.

Challenges we ran into

The biggest challenge was integrating the complex and cross-domain systems in little time. Over the last 36 hours, we learned and developed applications using the MindFlex glasses and AdHawk API, utilized object detection with YOLO (You Only Look Once) machine learning algorithms, and SORT object tracking algorithms.

Accomplishments that we're proud of

We got two major technologies working during this hackathon: the eye tracking and machine learning! Getting the MindLink to work with OpenCV was groundbreaking and enabled our machine learning algorithm to run directly using MindLink’s camera stream. Though the process was technically complex and complicated, we are proud that we are able to accomplish it in the end. We're also incredibly proud of the beautiful pitch deck. Our domain.com domain name (eyede.tech) is also a pun on "ID tech", "eye detect", and "I detect".

What we learned

In this hackathon, we had the privilege of working with the AdHawk team to bring the MindFlex eye tracking glasses to life. We learned so much about eye tracking APIs and their technologies as a whole. We also worked extensively with object detection algorithms like YOLO, OpenCV for image processing, and SORT algorithm for object tracking for temporal consistency.

What's next for Eyeware

Our platform is highly portable can be easily retrofitted to existing vehicles and be used by drivers around the world. We are excited to partner with OEMs and make this feature more widely available to people. Furthermore, we were limited by the mobile compute power at the hackathon, so removing that limit will also enable us to use more powerful object detection and tracking algorithms like deepSort.

Built With

Share this project:

Updates