NeuroClear

Landing Page_2
Landing Page_1
Screening page(Recording Started)
Progress page_2
Progress page_1
Screening page ( After Analysis )
Screening page(before analysis)
Screening page(Analysing the audio)
Screening page (Analysis report_3)
Screening page (Analysis report_1)
Screening page (Analysis report_2)
Practice page with the suggested exercises

Inspiration

Every impactful engineering project starts with a question that you simply cannot get out of your head. For me, that question arose during a visit to a hospital, where I happened to observe a speech therapy session. I watched patients working closely with therapists, practicing sounds, words, and different speaking exercises. I found myself wondering: what exactly makes a person need speech therapy?

That single question sent me down a rabbit hole. I discovered a motor speech disorder called dysarthria. Dysarthria occurs when the muscles involved in speech are affected, causing a person's speech to become slurred, slow, weak, or irregular in rhythm, even though they know exactly what they want to say.

But there was a secondary, equally critical problem. Speech therapy requires repeated sessions with a professional, which means regular travel and financial cost. A single session can cost around ₹700–₹1,200. The real engineering challenge became clear: how can we make assessment and guided practice more accessible between therapy sessions?

As a pianist, my brain is wired to listen carefully to sound—pitch, rhythm, timing, intensity, and subtle audio differences. As a student specializing in artificial intelligence and machine learning, my brain is wired to parse arrays, tensors, and neural networks. I realized I could combine my understanding of audio with machine learning to analyze speech, treating it as raw audio data containing patterns that a model could learn from.

What it does

NeuroClear is an AI-Assisted Dysarthria Speech Analysis and Exercise Platform. It acts as an accessible support companion for the therapy journey, bridging the gap between formal clinical appointments.

The core loop is simple: Record → Analyse → Practice → Record Again → Compare Progress.

When a user speaks into the application, the system extracts relevant speech and audio characteristics—analyzing clarity, pronunciation, timing, rhythm, and intensity. The AI then estimates whether the speech pattern is consistent with dysarthria.

Instead of stopping at a preliminary screening, NeuroClear asks, "What can I do now?". Based on the assessment, the application recommends personalized, guided speech exercises focusing on areas like breath control, articulation, and pacing. Finally, a longitudinal progress tracker maps the user's recovery metrics over time, providing valuable context for both the patient and their therapist.

How I built it

I built NeuroClear on a strictly decoupled architecture, separating the acoustic signal processing from the generative clinical reasoning.

1. The Frontend: I engineered a highly responsive, HIPAA-inspired Next.js UI using React and Tailwind CSS. The interface securely captures raw MediaRecorder audio streams directly from the browser, managing complex state hydration via local storage to ensure the user's progress and exercise regimens persist seamlessly across the application.

2. The ML Pipeline & DSP Backend: Audio files are sent to a Python FastAPI microservice. Instead of relying on basic speech-to-text, I utilized Hugging Face's Wav2Vec2 models to extract Connectionist Temporal Classification (CTC) log-probabilities. By mathematically analyzing the CTC loss—where the model seeks to maximize the probability of a target transcription $Y$ given an acoustic input sequence $X$ :

$P(Y | X) = \sum_{A \in \mathcal{A}_{X,Y}} \prod_{t=1}^T p_t(a_t | X)$

NeuroClear identifies micro-articulatory decay at the exact phoneme level. I paired this with custom SciPy digital signal processing (DSP) algorithms to extract amplitude envelopes, isolating involuntary motor tremors that standard language models entirely miss.

3. Generative Clinical Synthesis: The objective acoustic tensors are passed into the Gemini 3.6 Flash API using a strictly enforced JSON schema. Gemini digests these prosody and regression metrics to output structured diagnostic rationales and targeted rehabilitation protocols.

Challenges I ran into

Taking a complex machine learning pipeline from a local environment and keeping it alive on a constrained cloud server was our ultimate test.

1. Hardware Constraints: Loading PyTorch, Wav2Vec2, and Whisper into memory on a standard AWS EC2 Free Tier instance instantly triggered Linux Out Of Memory (OOM) crashes. I had to pivot our infrastructure strategy, ultimately discovering and migrating to an m7i-flex.large instance that provided the 8GB of physical RAM necessary for fast, concurrent tensor processing without relying on slow SSD swapfiles.

2. The Network Security Bridge: Vercel enforces strict HTTPS security protocols, which blocked our Next.js frontend from communicating with our AWS server's HTTP API endpoint (a Mixed Content block). We engineered a secure bridge by deploying an Ngrok tunnel directly on the EC2 instance, safely routing the payloads.

3. API Rate Limits: To protect the live demo from Gemini API limits (429 Too Many Requests), we implemented a robust Regex fallback system. If the AI pipeline is throttled, the backend gracefully catches the error and serves a simulated clinical payload, ensuring the UI never breaks during heavy judging traffic.

Accomplishments that I'm proud of

I am incredibly proud to have built a complete, end-to-end medical AI MVP application in exactly 13 days—from the initial repository initialization on August 2 to a fully deployed cloud architecture.

I successfully bridged the gap between complex audio signal processing and an intuitive, accessible user interface. Watching the system accurately extract a localized motor tremor from a raw audio wave and instantly map it to a dynamic "Speaking Rate" Recharts graph on the frontend was a massive technical victory.

What I learned

The most profound lesson from this project wasn't technical; it was clinical. I learned that AI should assist therapy, not replace the therapist.

When building healthcare AI, positioning the product accurately is vital. A machine-learning model can make predictions from speech patterns, but a true clinical diagnosis requires a qualified healthcare professional. We learned how to design a system that acts as an AI-assisted screening, exercise guidance, and progress monitoring tool—not an autonomous medical doctor.

What's next for NeuroClear

This MVP is just the foundation. To make NeuroClear a true companion for the therapy journey, we plan to focus on:

Multimodal Video Analysis: Integrating computer vision to track facial muscle movements, lip synchronization, and articulatory gestures during speech. Combining this visual data with our existing acoustic pipeline will yield a highly accurate, multi-dimensional assessment of motor deficits.
Interactive Therapy Tools: Embedding active clinical utilities directly into the practice dashboard. This includes tools like an interactive digital metronome to assist patients with pacing board drills and visual breath-pacing guides to help them maintain a steady speaking cadence.
Diverse Datasets: Training our models on larger, more diverse speech datasets to improve acoustic accuracy across varying demographics.
Global Accessibility: Expanding the feature extraction pipeline to support multiple languages and diverse accents.
Clinical Integration: Building dedicated therapist dashboards and real-time speech feedback mechanisms to integrate directly with professional speech therapy workflows.