EyeLearn

EyeLearn uses AI-powered gaze and blink tracking system to let users highlight text and learn hands-free, empowering people with limited mobility to access digital learning independently.

Comment

Inspiration

Over 1.71 billion people worldwide live with musculoskeletal or neuromuscular conditions that restrict hand movement. Traditional learning tools rely on typing and touching. We wanted to make digital learning truly inclusive by transforming natural eye movement into a new mode of control. The Assistive Techonology Market is growing with 5% of CAGR each year, and about 28% of them are visual aids or communication tool. We'd love to help individuals with hand mobility disability or paralysis enable independent learning without relying on manual input.

What it does

EyeLearn is an AI-powered web app that turns gaze and blink data into digital interaction. Users can highlight text, scroll, and generate flashcards hands-free. The system automatically detects focus points to create personalized study materials, enabling fully independent learning.

How we built it

For AI-powered gaze tracking, we used WebGazer.js to estimate real-time gaze coordinates from webcam input. For internal blink detection, we calculated Eye Aspect Ratio (EAR) to determine blink events. Then, we combined computer vision (MediaPipe) and AI regression models (WebGazer) in a unified web interface. We also executed multimodal signal fusion to create intuitive human-computer interaction.

Technical solution

1. AI-Powered Gaze Tracking Uses WebGazer.js to estimate real-time gaze coordinates from webcam input. Implements temporal smoothing (moving average) and low-pass filtering (α=0.22) for stable, natural motion.

2. Intentional Blink Detection Integrates MediaPipe FaceMesh to extract 3D facial landmarks. Calculates Eye Aspect Ratio (EAR) to determine blink events. Detects intentional blinks when eyes remain closed for >500 ms (EAR < 0.18).

3. Real-Time Learning Interaction Merges gaze coordinates and blink events to identify which sentence the user is focusing on. Automatically highlights text and generates flashcards based on gaze attention.

Challenges we ran into

1. Thresholding & Calibration Accuracy: Accurate gaze mapping was challenging due to inconsistent thresholding, lighting, and head-pose bias, requiring frequent recalibration.

2. Optimization & Smoothing: Applied moving-average and exponential smoothing filters to reduce jitter and balance real-time stability with low latency.

3. Debugging Interactive Behavior: Refined blink-event timing and asynchronous handling to ensure precise highlight detection and seamless UI transitions.

4. User Feedback & Calibration UI: Built a visual calibration interface with gaze-dot feedback and prompts to help users trust and validate tracking accuracy.

Accomplishments that we're proud of

  1. Helped people with limited hand mobility or paralysis achieve independent learning and reading.
  2. Developed AI-powered gaze tracking and intentional blink detection technology.
  3. Created meaningful social impact by improving accessibility for individuals with disabilities.

What we learned

1. Real-time interactivity : We learned how small timing issues or sensor noise can break interactivity — and how smoothing, calibration, and consistent feedback loops can fix it. 2. Importance of calibration : Accurate gaze tracking depends heavily on initial calibration, stable lighting, and consistent positioning.

What's next for EyeLearn

Product Development

1. Better Calibration Experience:

  • Add an on-screen calibration guide (e.g., “Click 5 points to calibrate”) and show feedback after each click to improve accuracy.

2. Enhanced Features:

  • Export flashcards as .txt or .csv
  • Add PDF export and modern CSS styling
  • Highlight individual words instead of full sentences for finer precision
  • Change the gaze dot color on blink events

3. Machine Learning Improvements:

  • Integrate head-pose detection (e.g., with MediaPipe FaceMesh) to correct for downward gaze bias.
  • Train a small regression model to fine-tune calibration data.

Business Development

1. Beta Testing with Target Users

  • Launch closed beta for individuals with musculoskeletal or neuromuscular conditions (e.g., ALS, spinal cord injury).
  • Partner with rehabilitation centers and accessibility organizations to collect structured usability feedback.
  • Measure engagement, ease of use, and independence improvement metrics.

2. Customer Interviews & Market Research

  • Conduct interviews with users, caregivers, and therapists to identify key pain points and learning behaviors.
  • Map competitive landscape and assess partnership opportunities with healthcare and education sectors.

3. Product GTM Plan

  • Initiate pilot programs in hospitals and educational institutions to demonstrate product value.
  • Scale through B2B collaborations, institutional licensing, and direct-to-consumer web integrations.
  • Position EyeLearn as the leading accessibility platform for digital learning and rehabilitation environments.

Built With

Share this project:

Updates