LooKey

🚀 Inspiration:

Communication is a vital part of the human experience, yet for individuals with neuromotor diseases such as ALS, expressing even basic needs can be a daily challenge. Our team was inspired to bridge this communication gap through modern technology. Guided by WeHack’s theme “Timeless Moments Await,” we designed LooKey, a web application that weaves together the past, present, and future to strengthen emotional bonds, ensure effective interaction, and adapt communication to evolving patient needs. With a bit of heart, some futuristic eye-tracking, and the raccoon as our mascot, LooKey brings hope to voice the voiceless.

🎉 What It Does:

• Eye-Tracking Quick-Access Phrases: Enables patients to pick what they want to say by gazing at predefined sentences, offering hands-free, intuitive text input.
• Confirmation Flow: After selecting a phrase by gaze, users are presented with a confirmation screen—left side = yes, right side = no. If confirmed, the phrase is spoken aloud.
• Text-to-Speech & Accessibility: Converts confirmed messages to voice. Supports customizable font sizes, color schemes, and keyboard layouts.
• Patient Dashboard: Allows patients to view their health notes and access the LooKey eye-tracker.
• Caretaker Dashboard: Enables caretakers to manage their patients—view info, monitor health logs, and add/remove patients as needed.

🛠️ How We Built It:

• Frontend: Built with React + Vite for fast, responsive, and accessible user interfaces.
• Backend: Developed using Node.js + Express, handling authentication, patient and caretaker data, health logs, and system logic.
• APIs/Tools:
  • WebGazer.js for real-time, browser-based eye-tracking.
  • Text-to-Speech (TTS) APIs for vocalizing selected phrases.
  • PostgreSQL to store and manage key data such as patient records, caretaker details, health records, and user preferences.

🚧 Challenges We Ran Into:

• Calibrating Eye-Tracking: Fine-tuning gaze detection for individuals with varying levels of motor control was time-intensive. Calibration needed to be highly personalized.
• Intentional Fixation Detection: We built a custom logic layer to differentiate between unintentional glances and deliberate gaze actions. Timing thresholds had to be carefully balanced.
• Lighting and Environmental Sensitivity: Eye-tracking accuracy was heavily influenced by lighting and camera positioning, demanding frequent environmental calibration.
• Micro-Movement Sensitivity: Slight head or eye shifts could throw off tracking, requiring the system to be incredibly precise and adaptive over extended use periods.

🏆 Accomplishments That We're Proud Of:

• Created a functioning eye-tracker that includes calibration, phrase selection, confirmation, and voice output.
• Designed two specialized dashboards—one for patients and one for caretakers—tailored to their needs.
• Built a scalable backend infrastructure that can expand with more advanced features and users in the future.

📚 What We Learned:

• Designing eye-tracking interfaces is as much about empathy as it is about precision.
• Calibration is not a one-size-fits-all process—each user journey is different.
• Thoughtful UX and accessibility considerations are essential in building assistive tech with dignity.

🔮 What's Next for LooKey:

• Multilingual Support: Expanding to support global users and diverse language needs.
• AI-Powered Routine Prediction: Learning from usage patterns to anticipate patient needs.
• Smart Notifications: Allowing caretakers to set triggers/alerts based on health data or user behavior.
• Blink Calibration: Differentiating between blinks as fatigue vs. confirmation gestures.

💻 Tech Stack:

• Frontend: React.js (with Vite)
• Backend: Node.js, Express
• APIs/Tools: WebGazer.js, Text-to-Speech APIs, & PostgreSQL

Built With

• express.js
• node.js
• react.js
• text-to-speech
• webgazer