Inspiration

The project was inspired by the case of a patient with ALS. This patient, like many with the condition, has motor deficits in the right hand, inhibiting them from writing, cooking, and living their old lifestyle. Our goal with the project was to build a device that allows a person to interact with the physical world using their eyes. Eye tracking felt natural and intuitive, and is spared with ALS.

What it does

eyeCatcher is a gripper and releaser that is controlled by eye tracking. When the user looks at a target, the system detects their gaze and triggers the gripper to perform a function. Within a specified timeframe, the number of times the user blinks corresponds to an action: 2 blinks will extend the fingers (to release), 3 blinks will contract the fingers (to grip), and 4 blinks will release entirely (failsafe). The result of this is a hand-free grasping system that responds to the user's command. This is a solid step towards more accessible assistive devices.

How we built it

The project consisted of two components: the eye tracker and the gripper mechanism. Using a webcam, we used eye tracking to capture the number of blinks and map it to a user command. These signals were sent via bluetooth to an ESP32 that drives the gripper's servo to open and close. Next, we designed and assembled a gripping mechanism (resembling a glove) using 3D printed components. On either side of the glove, we placed elastics to control the contraction and relaxation. Integrating and tuning these aspects culminated in our final version of the project.

Challenges we ran into

The biggest challenge was integrating and communicating between alterations in the mechanical design and software. Determining the correct angle for the servo motor to rotate was an important consideration, requiring various iterations, testing, and debugging. Additionally, we had to tell the difference between intentional blinks and natural blinks so that the gripper wouldn't trigger by accident. On top of that, we were working under tight time limits and had to simplify some of our original ideas.

Accomplishments that we're proud of

We built a working prototype that connects blinking to real physical movement. Seeing the gripper respond to our blinks for the first time was a huge moment. We're proud that we got both the hardware and software integrated within the timeframe. Most of all, we are proud that the project we have built has the potential to impact real people with limited mobility in their hand.

What we learned

  • Learned how eye tracking works and how to process blinks and turn them into signals for hardware.
  • Gained experience with integrating software and mechanical components, including sensors, actuators, and microcontrollers.
  • Learned about the iterative design process and how to debug software and prototypes.
  • Working as a team also taught us the importance of delegating and troubleshooting under pressure.

What's next for eyeCatcher

Currently, the device actuates to a predetermined angle in the code, which is okay for one user, but can pose problems when scaling up. Next steps would be to set up a calibration period at the beginning of the session, determining the adequate contraction and relaxation states.

Overall, our big-picture vision is a simple, hands-free system where a quick blink contracts or releases the gripper, with a built-in failsafe that automatically loosens the grip if it’s too tight or causes discomfort, and a camera that follows the user instead of staying fixed in place.

Built With

Share this project:

Updates

Submission history