EYEs.py

Introducing EYEs.py, a tool that uses eye-tracking glasses to capture what people are looking at on a webpage and analyzing that data to gain insights on user behaviour and how they interact with it.

Comment

Inspiration

We live in a society where data is extremely important. Technology all around us is constantly collecting it and it's generally being used to improve all our lives. Data allows us to make objectively better decisions guided by information. One relatively new way to gather data is through eye-tracking. Eye-tracking has many possible applications, including determining what a user is focusing on. Based on our experience with web development, we understand how tedious it is to get data on how your user is using your website. For example, are they focusing on what we want them to? Or are they spending time getting distracted by other content on the page? Up until now there hasn't been a great way to get this info, but now there is.

EYEs.py is an innovative solution to collect data through tracking eye movements. Compared to alternative methods of data collection, this approach allows developers to get data on user behaviour by tracking how they do things, rather than just what they do. This new analysis method is referred to by industry as behavioural analytics and is growing rapidly. Specifically, EYEs.py takes where the user is looking and creates a heat map from it to determine what the major areas of focus are on the webpage.

What It Does

The AdHawk MiniLink's eye tracking sensors track the position of the eye and send it through the tethered USB connection to the computer to AdHawk Backend. From here the data travels through the AdHawk Python SDK and into our Python code where we populate a matrix with values from the eyesight tracking. These values are then used for the heat map with the darker areas having a higher amount of time spent looking at them relative to the light areas. To only capture eye movements on the screen, the forward facing camera on the glasses tracks the 4 markers on the corners of the page. Utilizing this allows the glasses to determine when the screens is being looked at.

How We Built It

We built this project utilizing the AdHawk suite of products:

  • AdHawk Microsystems MindLink glasses
  • Utilized the AdHawkAPI to enable data collection
  • AdHawk Python Software Development Kit (SDK)
  • Object-oriented Python code

Challenges We Ran Into

  • Initially, we struggled to collect and store the data from the glasses through the API provided by AdHawk
  • We had many issues with working with QT (the GUI system) that was already being used by Adhawk regarding formatting
  • We had issues connecting to CockroachDB and sending data over

Accomplishments That We're Proud Of

Most of us our team is first-time hackers, and we felt proud to complete a MVP (Minimum Viable Product) of an idea that we were all interested in building. It was also super cool to work with the AdHawk MindLink glasses as this technology was truly cutting edge and a rare opportunity. More importantly, we are proud of what we were able to accomplish with a completely foreign technology in roughly 48 hours.

What We Learned

This project taught us many valuable lessons including:

  • Organization (keeping track of everyone's tasks and their status)
  • Collaboration (how to effectively utilize Git to work together efficiently)
  • Dedication (persevering through a lack of sleep)

Additionally, it helped us improve our coding skills too especially with technologies such as:

  • Python3
  • CockroachDB (PostGreSQL)
  • Github

What's Next for EYEs.py

One of our favourite aspects of this projects is its potential. There are many features that we believe could be added to improve the usability, versatility, and efficacy, like:

  • Create a recording tool to display the order in which each part of the website was viewed (i.e. the user first viewed the top left part of the page)
  • Implementing a web-based front end with an API communication layer with Python backend
  • Adapting our heat maps to adjust with scrolling and changing routes in a website
  • Creating a dashboard for managing different user recordings and grouping the data
  • Rescaling to prevent the tracking markers from covering content
  • Combine insights with Microsoft Clarity and Google Analytics to get even more data on website performance for a wider variety of scenarios
  • Train an AI/machine learning model to suggest changes by noticing pain points in the UI

Built With

Share this project:

Updates