Inspiration
A 2022 report from Statistics Canada indicated that 5% of Canadians aged 15 years or older (over 1.5 million people) had a dexterity disability. Our team thought, how can we design a hands free navigation system that allows these people to use a computer, something that we use everyday?
What it does
Our solution is Gaze, a hands-free computer navigation system that utilizes EMG processing and Computer Vision to track a users gaze and eye blinks to control their mouse.
How we built it
EMG processing is handled using Uspide Down Labs' Bioamp EXG Pill connected to an Arduino Uno. Electrodes placed near the eyebrows send EMG signals from muscle contractions, which are then processed through the Arduino to send actions to the computer. Currently, blinking is set to mouse clicks. Gaze detection is handled using L2CS-Net, an open-source, efficient model for gaze estimation and tracking, and deployed using OpenCV. Normalized coordinates of the users' gaze are returned, which are then used to move the mouse cursor to where the user is looking.
Challenges we ran into
A tough challenge we ran into was determining a way to efficiently handle eye tracking. Our first iteration of our eye tracking program utilized a less robust method of detecting the pupils, and returning a positive or negative value depending on the direction the user was looking (up/right - positive, left/down - negative). While this method did work with our the rest of our code, it gave inconsistent results, so we decided to search for other alternatives, which eventually led us to look into gaze detection models.
Accomplishments that we're proud of
Accomplishments we're proud of are the accuracy of our mouse control, our team worked to ensure that eye movement is mapped to cursor position as best as we can make it, and eye blinks are registered with low latency. Additionally, we're proud that we were actually able to stick with our plan and build a working prototype compared to our first hackathon that we participated in (we don't talk about it).
What we learned
Our team gained valuable experience working with hardware and biosensing technology, as well as utilizing computer vision models.
What's next for SixSeven Solutions - Gaze
Future development for our project includes:
- Implementing typing control
- Refining our gaze detection program
- Switching to a more optimized and higher performance codebase
Log in or sign up for Devpost to join the conversation.