Inquire

Inquire logo

Inspiration

We were inspired by a clear gap in how cognitive decline is currently monitored. Cognitive assessments for conditions like early dementia or Parkinson’s-related cognitive impairment typically take place during in-person clinical visits and are often administered manually. Because these evaluations are tied to scheduled appointments, which may occur annually or at periodic follow-ups, subtle changes in speech fluency, coordination, or attention can go unnoticed between visits. We saw this firsthand with our own grandparents, where small shifts in speech patterns and reaction time were noticeable to family members long before they were formally evaluated in a clinic. We also observed that screening can require appointments, transportation, and clinical resources that are not always easily accessible, especially for older adults. That experience motivated us to explore whether cognitive screening could become more accessible using tools people already have, such as a laptop and webcam.

Another key inspiration was the idea that cognition is not a single signal but a combination of observable behaviors. It appears in speech patterns, motor coordination, and eye movement control. Instead of building a single-task assessment, we designed a multimodal tool that guides users through speech, movement, and eye tracking tasks in one structured session. By automatically recording, transcribing, scoring, and storing results, the system creates objective signals that can be tracked over time. Our goal was to move cognitive assessment from something episodic and subjective to something repeatable, automated, and accessible from anywhere.

What it does

Inquire runs three clinical neurological tests right in your browser using just your webcam and microphone. No downloads, no apps, no external AI services.

Speech Test (Phonemic Verbal Fluency)
You get 60 seconds to say as many words as you can that start with a random letter. This is based on the FAS test that neurologists use to check frontal lobe function.

How it works:
The browser's speech recognition listens in real time. Words appear as colored badges: green for matches, yellow for repetitions (which doctors call perseveration), gray strikethrough for non-matches.

Hand Movement Test (Luria Sequencing)
You perform a hand sequence three times: fist, karate chop edge, palm down flat. This checks motor planning and sequencing ability.

How it works:
MediaPipe tracks 21 points on your hand at 30 frames per second and draws a skeleton overlay. It detects each pose using geometry: fist (fingertips near palm), edge (hand vertical), palm down (hand horizontal with fingers extended). It waits for 6 consistent frames before accepting each pose.

Eye Tracking Test (Smooth Pursuit)
A dot moves around your screen for 15 seconds. You follow it with your eyes while keeping your head still.

How it works:
MediaPipe FaceLandmarker tracks 468 facial landmarks including your iris positions (landmarks 468 and 473 for precise eye tracking). The system calculates your gaze direction by measuring the position of your iris relative to your eye corners. The dot turns green when you're on target, red when you're not. The system measures gaze accuracy and movement smoothness between frames.

Your Results
You get a dashboard with scores, charts, and personalized recommendations. The overall score averages your three tests. Above 80 is normal, below 40 suggests seeing a doctor.

What you see:
Radar chart comparing all three scores, bar chart breakdown, hand position scatter plot (tighter = better stability), and eye movement line graph (lower = smoother). Each result includes plain explanations plus research citations you can expand.

Recommendations:
The system generates specific advice: word games for low speech scores, hand exercises for motor issues, when to see specialists. Each is tagged as exercise, action, or info.

Additional features:
User accounts with secure authentication, assessment history tracking, PDF report generation, admin dashboard, and it works with glasses thanks to multiple facial reference points.

How we built it

We used Next.js and React for the web app with TypeScript to keep everything organized. For the AI stuff, we integrated MediaPipe which does the heavy lifting for hand and face tracking. The speech recognition uses the browser's built-in Web Speech API.

On the backend, we set up authentication with JWT tokens, used Drizzle ORM with SQLite for the database, and built API routes for everything. We also added video recording during tests and PDF generation for the reports. The whole thing is styled with TailwindCSS to keep it clean and easy to use, especially for older folks.

Challenges we ran into

Getting MediaPipe to work smoothly was rough. It kept throwing strange console errors, and we had to figure out how to suppress them without breaking the pipeline. The eye tracking was extremely inaccurate at first until we realized we needed to track relative iris positions instead of absolute ones.

Making the hand tracking recognize poses correctly also took a long time. We had to tune several thresholds and add debouncing so it would not register false positives. Running all of this AI processing in real time while recording video significantly reduced our frame rate, so we had to optimize carefully to keep the experience smooth.

The hardest part was making the system simple enough for elderly users. We kept removing features and simplifying the interface based on feedback. What made sense to us as developers did not always make sense to our grandparents.

Accomplishments that we're proud of

We actually got real clinical tests working in a browser with just a webcam! The hand tracking can accurately detect three different hand poses in real-time, and the eye tracking works even with glasses, which was a huge win.

Building a complete platform with auth, database, admin panel, and PDF reports in such a short time feels pretty awesome. But honestly, the coolest part is that this could actually help people catch serious conditions early without needing expensive equipment or doctor visits.

What we learned

We learned a great deal about how cognitive and neurological assessments actually work and why clinicians rely on specific structured tests. Working with MediaPipe showed us that browser-based computer vision is far more capable than we initially expected, but it also revealed clear limitations in accuracy and performance under real-time constraints.

The biggest lesson was about designing for accessibility. We had to completely rethink our interface after realizing that elderly users interact with technology very differently than we do. We also gained a deeper understanding of the ethical responsibility involved in building health-related tools. It became clear that this system must be positioned as a screening and monitoring aid rather than a diagnostic instrument.

What's next for Inquire

First, we want to validate the system with neurologists and primary care physicians to ensure that our scoring aligns with established cognitive assessment standards. We would also like to expand the platform by adding structured memory tasks and reaction time assessments to create a more comprehensive cognitive screening experience.

Long term, we plan to build a mobile application, add multilingual support, and implement longitudinal trend tracking so users and clinicians can monitor changes over time. Integrating with telehealth platforms would allow results to be shared directly with healthcare providers, supporting more informed conversations during appointments.

Our long-term vision is to make Inquire a widely accessible screening and monitoring tool that helps identify early signs of cognitive change and supports timely medical evaluation.