Inspiration
I'm currently rewatching my Systems Programming lectures to prep for the midterm. Sometimes, it's easy to get distracted and tune out what I'm watching though. I wanted to build a machine learning-based application that tracks your eye movement to determine if you're really paying attention during online lectures.
What it does
It's a fully functional web app that lets you record eye-tracking data, make predictions, and visualize some cool results!
How we built it
I hosted all my code on Github. The app was built entirely in Python and hosted with Streamlit.
Hardware was from Tobii! I used their SDK extensively in order to build this.
In order to create the actual machine learning model, I recorded me actively watching a Systems lecture, and another with me not paying attention at all (zoning out and even having conversations with my parents during the lecture). I used samples from both of these recordings to build up an original dataset, convert the data into useful features (like velocity of eye movements) and train a XGBoost model on it.
Challenges we ran into
First time using Tobii! I had some difficulties getting started, and figuring out how to stream data from the device.
Accomplishments that we're proud of
This was so cool!! I found out some pretty interesting stuff about my eyes, and about eyes in general. Apparently, one of my pupils is consistently larger than the other one (110% the other's size). And, I have slower eye movements when I'm paying attention vs not paying attention.
Here's a cool correlation matrix I plotted!
What we learned
- how to develop with Tobii and Streamlit together
- eyes are really cool, and apparently have super reliable patterns
What's next for Innovision
I want to see if I can try predicting other things (not just attentiveness) and add more visualizations!
Log in or sign up for Devpost to join the conversation.