Nöra - Cognitive Screening AI Assistant

An autonomous voice AI agent conducting real-time cognitive screenings to democratize early detection of Alzheimer's risk through natural, zero-hallucination conversations

Comment

Inspiration

Early-stage cognitive decline—including Alzheimer's disease—is notoriously difficult to detect. Most people only seek medical help after symptoms become severe and largely irreversible. Traditional clinical screenings are expensive, require trained personnel, and are completely inaccessible to millions. We asked a critical question: What if an AI agent could conduct a preliminary, clinical-grade cognitive screening through natural voice conversation—accessible to anyone, anywhere, at any time? Nöra was born from the intersection of applied Generative AI and a deep societal need. We wanted to prove that real-time multimodal AI can go far beyond chatbots and content generation; it can genuinely assist in proactive healthcare and democratize early detection.

What it does

Nöra is an autonomous voice AI agent that conducts a structured cognitive screening session with the user through natural, real-time conversation. It administers 4 clinically-inspired cognitive tests:

  1. Verbal Fluency Test: The user names as many words as possible in a specific category within a time limit.
  2. Story Recall Test: A dynamically generated story is told; the user must retell it from memory.
  3. Visual Recognition Test: Unique, AI-generated images are shown, and the user identifies the objects.
  4. Orientation Test: Time, date, and spatial awareness questions combined with real-time camera analysis for focus and presence.

At the end of the session, Nöra algorithmically produces a risk assessment score and a detailed, downloadable PDF report with per-test breakdowns.

How we built it

We engineered a sophisticated, production-ready system to ensure clinical reliability:

  • Core GenAI: Gemini Live API for real-time, interruptible voice conversation via a custom WebSocket proxy, and Imagen 4 for dynamic test image generation to prevent memorization.
  • Multi-Agent Orchestration: We utilized 5 coordinated agents (Nöra, BrainAgent, VisualTestAgent, DateTimeAgent, VideoAnalysisAgent) managed by a centralized state machine.
  • Deterministic Scoring (Zero Hallucination): The LLM is only used for natural interaction and data collection via tool calling. All score calculations happen algorithmically in the Node.js backend, completely eliminating hallucination risk in health assessments.
  • Infrastructure: React + Vite + TailwindCSS frontend with AudioWorklet for low-latency audio. Node.js backend with PostgreSQL, fully automated via IaC and deployed on Google Cloud Run.

Challenges we ran into

  1. LLM Hallucination in Scoring: Early experiments showed the AI miscalculating test scores. We solved this by separating reasoning from math: implementing deterministic scoring where the AI only collects data via tool calling, and backend algorithms do the math.
  2. Real-Time Audio Pipeline: Browsers cannot directly connect to the Gemini WebSocket endpoint due to CORS constraints. We built a custom Node.js WebSocket proxy and utilized AudioWorklet for sub-second latency audio streaming without memory leaks.
  3. Timer Management for AI: LLMs have no reliable sense of time. We shifted all timer logic to the backend with server-side setTimeout, injecting timer events into the AI's context dynamically.
  4. Multi-Agent Race Conditions: Managing 5 concurrent agents required careful state synchronization.

Accomplishments that we're proud of

  • Zero-Hallucination Scoring: By enforcing strict architectural boundaries between AI data collection and algorithmic computation, we achieved 100% reliable test scoring.
  • True Real-Time Voice Interaction: We built a sub-second latency voice agent with interrupt support—a true conversation, not just a turn-based chatbot.
  • Dynamic Test Generation: Every single session generates unique stories and images, ensuring users cannot "game" or memorize the test.
  • Genuine Societal Impact: We built a highly scalable, production-ready application that could realistically democratize access to cognitive screening worldwide.

What we learned

  • Real-time voice AI requires extraordinarily explicit system instructions to maintain a consistent persona across a long, multi-step session.
  • Tool Calling is incredibly powerful for structured data collection, but requires aggressive server-side sanitization to prevent session crashes from malformed JSON.
  • Separating AI reasoning from deterministic computation is the absolute golden rule for building reliable AI applications in healthcare.

What's next for Nöra - Cognitive Screening AI Assistant

  • Clinical Validation: Partnering with neurologists to benchmark Nöra's screening accuracy against established clinical tests like MoCA and MMSE.
  • Longitudinal Tracking: Allowing users to retake screenings over time to detect subtle downward trends in cognitive health.
  • Multilingual Expansion: Scaling beyond the current languages to support global accessibility.
  • On-Device Processing: Exploring edge AI deployments for maximum privacy in sensitive health environments.

Built With

Share this project:

Updates