Eye AAC Select: Adaptive AAC Eye Tracking System

Inspiration

Approximately 97 million people around the world use AAC, or Augmentative and Alternative Communication. These devices allow people without a voice to speak. However, they universally have 2 problems: access and dehumanization. AAC can be extremely expensive, and many programs don't properly accommodate physical disabilities. Additionally, these devices often use robotic voices that may be dysphoric, dysphoric, or otherwise uncomfortable. Our project aims to address both of these problems.

What it does

Our project is a web-based AAC program with built-in eye-tracking controls and a realistic voice provided by ElevenLabs. It includes gorgeous and descriptive icons, a friendly user interface, and an instantly understandable calibration system. Additionally, it includes a predictive word generation system powered by AI, which provides icons to select from a list of the most likely follow-up words.

How we built it

We utilized the open-source library EyeGestures Lite to provide local eye tracking, and the program itself was made in TypeScript with the help of ChatGPT and Claude Code. The icons were created by a team member in IbisPaint.

Challenges we ran into

Figuring out eye tracking was a significant challenge and was the source of the majority of our work and errors. Additionally, we worked asynchronously, which caused some issues when trying to merge our code on GitHub.

Accomplishments that we're proud of

We're extremely proud of our eye tracking technology and interface. Additionally, we're extremely excited by the possibilities that this technology provides, as we can help our community in a real and impactful way.

What we learned

We learned a lot as highschoolers who have only been to a few hackathons. For many of us, this was our first time using TypeScript and integrating our code with AI tools. We're now much more confident in these areas.