NeuroBridge SenseAssist

Inspiration

NeuroBridge SenseAssist was inspired by the gap between clinical rehabilitation sessions and everyday patient practice. Many patients recovering from stroke and motor speech disorders lack continuous guidance at home, leading to repeated mistakes and slower improvement. We wanted to build an intelligent companion that can observe, understand, and adapt like a personalized therapy assistant.

What it does

NeuroBridge SenseAssist is an autonomous multi-agent neuro-rehabilitation system that analyzes speech patterns and facial motor movements to identify articulation difficulties. It uses AI reasoning to understand the underlying issue and recommends personalized sensory interventions such as rhythmic haptic pacing and visual articulation cues. The system creates a closed-loop cycle where patients practice, receive adaptive feedback, and track progress.

How we built it

We built NeuroBridge using a multi-agent AI architecture combining:

• Speech Perception Agent for phoneme errors, pauses, and speech characteristics.
• PulseSight Facial Perception Agent for facial motor and articulation analysis.
• Gemini-powered Reasoning Agent for multimodal interpretation and decision-making.
• Therapy Optimization Agent for adaptive intervention selection.
• Safety Boundary Agent for fail-closed safety checks.
• Digital Twin Patient Model for progress tracking.

The prototype integrates multimodal sensing, AI reasoning, and rehabilitation workflow visualization into a unified platform.

Challenges we ran into

The biggest challenge was designing a system that goes beyond simple speech recognition. Understanding the difference between detecting an error and reasoning about its cause required combining multiple data sources.

We also faced challenges in creating explainable AI decisions, designing safe intervention boundaries, and developing a realistic closed-loop rehabilitation workflow.

Accomplishments that we're proud of

We are proud of creating a complete agentic healthcare prototype that demonstrates:

• Multimodal speech and facial analysis.
• Autonomous AI reasoning using specialized agents.
• Adaptive sensory feedback recommendation.
• Safety-aware intervention control.
• Transparent agent decision tracing.
• A personalized rehabilitation workflow.

NeuroBridge demonstrates how AI can extend rehabilitation support beyond traditional clinical environments.

What we learned

Through this project, we learned that healthcare AI requires more than accurate predictions. A successful system must be explainable, safe, user-centered, and designed around real clinical workflows.

We also learned the importance of combining different AI capabilities—perception, reasoning, and adaptation—to create meaningful solutions.

What's next for NeuroBridge SenseAssist

Our next steps are to validate NeuroBridge with real patient data, improve personalized adaptation using larger rehabilitation datasets, integrate wearable sensing devices, and develop a clinically deployable platform with stronger interoperability and therapist collaboration.

Our vision is to create an accessible AI rehabilitation companion that supports patients anytime, anywhere.

Built With

• claude-sonnet
• esp32-ble
• feedback
• fhir-r4
• firebase-firestore
• gemini-3.7
• gpt-5.6
• haptic
• mediapipe
• multi-agent-ai-architecture
• next.js
• python
• react
• typescript