NeuroEcho

NeuroEcho — “When Your Brain Says Rest, NeuroEcho Listens.”

Comment

Inspiration

Concussion recovery requires a careful balance between cognitive activity and rest. We were inspired by a simple question: What if a cognitive training platform could recognize signs of increasing fatigue and encourage rest instead of pushing the user to continue?

This led us to build NeuroEcho, a privacy-first, browser-based cognitive rehabilitation support prototype designed with photophobia and cognitive fatigue in mind. Our goal was to shift the focus from engagement to protective pacing—listening to changes in the user's performance and eye-movement behavior and responding with a safety-first intervention.

What it does

NeuroEcho combines low-stimulation cognitive exercises with real-time, on-device eye tracking.

It includes three mini-games:

  • Orbit Velocity — tests sustained visual attention and reaction time.
  • Echo Recall — tests working memory through reverse-order sequence recall.
  • Chrono Tap — tests response inhibition using a go/no-go mechanism.

The system tracks behavioral signals such as reaction time and correctness while MediaPipe FaceLandmarker analyzes facial and iris landmarks locally in the browser.

These signals contribute to an explainable fatigue-risk score based on reaction-time degradation, error rate, gaze jitter, and reaction-time volatility.

The system classifies the user into STABLE, CAUTION, or HIGH_RISK states. When HIGH_RISK is reached, NeuroEcho activates a safety latch and presents a low-stimulation rest screen instead of encouraging the user to continue.

Importantly, NeuroEcho is a research/hackathon prototype, not a medical device, and its fatigue score is a hand-tuned heuristic rather than a clinically validated measurement.

How we built it

NeuroEcho is built as a single-page React application using:

  • React 18 — frontend framework
  • Vite 5 — development and build tooling
  • Tailwind CSS 3 — low-stimulation UI
  • MediaPipe FaceLandmarker — client-side facial and iris landmark detection
  • Web Audio API — gentle audio feedback for Echo Recall
  • React Context + useReducer — centralized state and fatigue engine
  • SVG + DOM/CSS — lightweight game visuals and animations

The webcam processing happens locally in the browser. We extract iris landmarks, normalize gaze relative to the user's eye geometry, calculate gaze jitter using a rolling statistical window, and combine these signals with gameplay performance.

The fatigue engine uses a weighted heuristic:

[ Score = 100(0.35R + 0.30E + 0.20G + 0.15V) ]

where the components represent normalized reaction-time slope, error rate, gaze jitter, and reaction-time volatility. The score is smoothed using an exponential moving average to avoid sudden fluctuations.

We also built a truthful session-reporting system where metrics are taken directly from the session event log. If a reaction time cannot honestly be measured, the application displays "Not measured" instead of fabricating a value.

Challenges we ran into

Real-time eye tracking

Webcam landmark data can be noisy, and absolute facial coordinates can be affected by head movement. We addressed this by normalizing iris positions relative to the user's own eye geometry.

Signal stability

Raw frame-by-frame measurements can fluctuate significantly. We implemented smoothing, rolling statistics, and incremental variance calculations to produce a more stable gaze-jitter signal.

Designing a meaningful fatigue signal

Combining reaction time, accuracy, gaze jitter, and volatility into one score required careful weighting. We deliberately kept the system explainable and clearly labeled the result as a heuristic rather than a clinical measurement.

Safety-first interaction design

We wanted the system to do more than display a warning. Once HIGH_RISK is reached, the safety trigger remains latched until explicitly acknowledged, preventing the interface from rapidly switching between playing and resting.

Low-stimulation accessibility

Because photophobia and cognitive fatigue were important design considerations, we avoided flashing effects, abrupt transitions, and highly saturated visuals. We also incorporated reduced-motion support, keyboard controls, ARIA labels, visible focus states, and optional gentle audio.

Accomplishments that we're proud of

We are proud that NeuroEcho goes beyond being a collection of mini-games and demonstrates a complete adaptive safety loop:

Observe → Analyze → Estimate → Explain → Protect

The project successfully combines:

  • Real-time browser-based eye tracking
  • Behavioral performance analysis
  • Explainable fatigue-risk scoring
  • Adaptive cognitive exercises
  • Automatic rest intervention
  • Privacy-first on-device processing
  • Truthful session analytics
  • Low-stimulation and accessibility-focused design

We are especially proud of the decision to make protective pacing the core interaction principle rather than maximizing session duration.

What we learned

We learned that building a health-oriented technology prototype requires more than technical implementation. It requires careful thinking about privacy, explainability, accessibility, limitations, and responsible claims.

Technically, we learned how to work with facial and iris landmarks, streaming statistics, exponential moving averages, behavioral metrics, and state-driven safety mechanisms.

We also learned an important lesson: a prototype should be honest about what it can and cannot prove. NeuroEcho's fatigue model has hand-tuned weights and thresholds and has not been validated against clinical concussion outcome data.

What's next for NeuroEcho

The next stage would be to move NeuroEcho from a hackathon prototype toward a more rigorously validated research platform.

Potential next steps include:

  • Conducting controlled usability studies
  • Collecting appropriately consented, anonymized research data
  • Validating fatigue indicators against expert/clinical assessments
  • Exploring personalized baseline calibration
  • Improving robustness across different cameras, lighting conditions, and users
  • Adding more cognitive exercises
  • Improving accessibility and personalization
  • Investigating more advanced statistical or machine-learning models
  • Adding stronger privacy-preserving analytics
  • Exploring clinician/researcher dashboards for authorized use

The long-term vision is to create technology that adapts to the person rather than asking the person to adapt to the technology.

NeuroEcho — Listen to Your Brain. Recover at Your Pace. [link]- (https://neuroecho.onrender.com)

Built With

  • ai
  • api
  • cognitive
  • computer
  • context
  • css
  • data
  • eye
  • face
  • healthcare
  • javascript
  • mediapipe
  • ml
  • on-device
  • react
  • svg
  • tailwind
  • usereducer
  • vision
  • visualization
  • web
  • webrtc
Share this project:

Updates