-
What HawkEyes sees
-
Planned Feature - Phonetic Highlighting
Inspiration
One of our teammates has Dyslexia, and noticed that a major hurtle to his reading proficiency was losing track of his place on a page. Through researching the different ways in which Dyslexic’s eyes move across a page, we determined a gaze tracking approach could solve this problem.
What it does
HawkEyes helps Dyslexics keep a sharp eye on their reading. It’s a Google Chrome extension that tracks the user’s eyes to see if they’ve lost their place and leaves a bookmark on the last word they read, so they can pick up quickly where they left off.
How we built it
The program gathers training data by correlating images of the user’s eyes with where they click on the screen. Then, a machine learning algorithm and OpenCV are able to predict where on the screen the user is looking. The software detects the characteristic eye movement of the reader’s gaze drifting and the word that the user was on highlights.
Challenges we ran into
Finding classification elements for pupils, connecting the server and display client with web sockets, integrating with Tobii SDK.
Accomplishments that we're proud of
Making a hack that one of our teammate will use (and hopefully a lot of other people, too!), finishing the hack on time, keeping our eyes focused enough to train the ML model no matter how tired we were.
What we learned
This was our first time using OpenCV, our first time building Chrome Extensions, our first exposure to machine learning, and, for one teammate, his first Hackathon!
What's next for HawkEyes
Bring this to multiple platforms, letting teachers gather data about how students’ reading performance, making it easy to set up, improving tacking accuracy, detecting if a user is stuck on a word and showing the word’s phonic spelling.
Log in or sign up for Devpost to join the conversation.