ALSxaPlus

An Alexa+ add-on for people with speech disabilities: it offers sentences for unclear speech, speaks the one you choose, and learns your voice over time.

Comment

Inspiration

Voice assistants are built for typical speech. People whose speech is changed by ALS, cerebral palsy, Parkinson's disease or a stroke are misheard again and again. But a recogniser's mistakes for one person are usually consistent, and a pattern that repeats can be learned.

What it does

ALSxaPlus is an idea for an Alexa+ add-on, shown through a web simulation of a smart display that talks to a real MCP server. When speech looks unclear, it asks once, "Do you have difficulty speaking?", and only the person's own answer turns the help on. Then it shows one recommended sentence and five others, the person picks (by touch, keys or a single switch) or types, the sentence is said aloud in a clear voice, and the sounds and the chosen sentence are saved so the next guess is better.

How we built it

An MCP server written in Rust (the official rmcp SDK, five tools) with PostgreSQL. Claude Haiku on Amazon Bedrock proposes the order of the sentences, and code checks it: anything outside the offered list is rejected, plain rules take over if the model fails, and a monthly spending limit is enforced. Deepgram handles speech (three recognisers listen to the same recording, and Aura 2 speaks the answer). The simulation is built with React Router and checked for accessibility at every step. Claude Code with Claude Sonnet 5.5 on Amazon Bedrock was used as a Code Assistant to accelerate development and debugging.

Challenges we ran into

Forced model tool calls still returned empty or extra fields, Deepgram's newer models return only one transcript, and Alexa+ tool access is limited, so we simulated the display and tested the server with the official MCP client. We also chose not to "detect" disability: unclear speech has many causes, so the estimate only decides whether to ask.

Accomplishments that we're proud of

Control stays with the person: nothing is switched on without their answer, and they can see, forget or delete everything learned. On 640 recordings of 8 speakers with dysarthria (TORGO), offering the speaker's own sentences put the right one first 62% of the time, against 42% for the recogniser's best transcript alone.

What we learned

The model proposes and the code disposes: the model improves the order, the rules keep it safe. A forced tool call is not a guarantee, so everything is validated and a plain fallback always works.

What's next for ALSxaPlus

Work with people who live with these conditions, their carers and therapists (no one with a speech disability has used it yet), run it on Alexa+ once tool access is available, and test on free speech rather than read prompts.

Built With

Share this project:

Updates

Submission history