NeuroCue
Inspiration
Every 3 minutes, someone in the US loses their words to a brain injury (about 180,000 new cases a year, National Aphasia Association). More than 2 million Americans live with aphasia: they still think and feel, but the words won't come. Speech therapy helps, but nobody knows which therapy works best for which patient. A typical aphasia trial has about 53 people and takes years, and public speech data only says whether an answer was right or wrong, never what kind of mistake it was. Research can't move faster than the patients it can recruit.
What it does
NeuroCue creates virtual aphasia patients you can run experiments on. In the patient console you:
- Make a patient from two numbers: how severe, and where the damage is (meaning vs. sound).
- Hear her name real pictures and make the kinds of mistakes real patients make ("puzzle" → "pahshuw").
- Diagnose her: her hidden lesion is recovered from her answers alone.
- Treat her: the clinic's standard hint ladder vs. an AI therapist trained with reinforcement learning, on identical copies of her.
- Correct the therapist with DITTO: a clinician says "I would have…", and the AI adopts that rule only where she meant it.
How we built it
- Patient model: Dell's two-step model of word retrieval; our implementation reproduces Dell's 14 published patient fits.
- Real population: severity and lesion mix fitted to the PSST / AphasiaBank naming recordings (107 sessions), with 33 participants held out for testing.
- Real words: errors are turned into actual words and nonwords over 481 standard IPNP pictures.
- Diagnosis: a grid maximum-likelihood fitter recovers the two lesion weights from response types.
- Learning over days: a memory model fitted to Middleton et al. 2019 (spaced vs. massed practice; every condition within 8 points), with hint strengths calibrated to Nardo et al. 2017.
- AI therapist: MaskablePPO picks the word and the hint on every trial. On 80 held-out virtual patients it keeps 71% of practised words a week later, vs. 68% for the standard hint ladder, matching the best schedule we tested (spaced practice with few hints).
- Clinician in the loop: DITTO (Demonstration ITerated Task Optimization, Shaikh et al., ICLR 2025). Each clinician correction is treated as preferred over the AI's own choice, and the policy is retrained while anchored to its original self. In the demo, one correction changes about 10 of 160 decisions, all right after errors. A language-model version (Qwen3-1.7B with LoRA) is also in the repo as an exploratory check: on 10 reserved situations it agreed with the demonstrator 6/10 times after DITTO, vs 2/10 for the base model.
- Our addition: our clinical lead hand-coded 328 real PSST answers by error type, to check the simulated error mix against real patients.
Stack: Python, PyTorch, Stable-Baselines3 (MaskablePPO), Gymnasium, TRL/PEFT, FastAPI, Modal.
Challenges we ran into
- Making the simulator honest, not random. Every layer is fitted to published data and tested on held-out patients, and the console shows where each number comes from.
- Public data has no error types. PSST marks answers only right or wrong, so we built a coder and had our clinical lead adjudicate the uncertain cases.
- DITTO overgeneralising. Early corrections changed the policy everywhere. Anchoring to the original policy and spreading demonstrations across situations kept changes local.
What we learned
Small trials are the bottleneck in aphasia research. A validated simulator can test thousands of therapy plans before a single patient is asked. Reinforcement learning finds what works on average; clinicians know what works for this patient, and DITTO lets them teach it.
What's next
Train student clinicians on virtual patients before real ones, check each patient's personalised plan on her virtual twin first, and validate with speech therapists and aphasia labs.
All results are in simulation. NeuroCue does not diagnose and is not clinically validated. It is fitted to public research data (PSST/AphasiaBank, Middleton 2019, IPNP); no private patient data was used.
