Iris Speak

Product in action (Card generation and selection)
Welcome page (Dashboard)
Parent turn (Enter in sentence through voice or text)
Child turn (Card selection screen)
Child turn cont. (Sentence generation screen)
Visual accessibility options
Conversation transcript and rating
Profile (Settings)
Personalized Vocabulary (Settings)
Past conversations (Settings)

Inspiration

While volunteering at a local organization helping autistic children, we noticed how it was difficult to communicate with those who were nonverbal. Some of them had AACs, but we found real flaws in the device: communication was slow and took the child lots of effort to look through the cards just to convey a single thought, and there was little to no personalization.

What it does

To combat these problems, we created Iris Speak: a context-aware AAC app that displays only the relevant cards, reducing overstimulation, speeding up communication, and personalizing vocabulary to each child’s unique profile. Iris Speak also simulates the typical conversational format, as children and parents take turns speaking. The parent enters a sentence or question through speech or text, and the child then selects their chosen cards. Once a child selects the cards for a sentence, Iris Speak generates a full sentence and reads it out loud.

How we built it

Throughout the entire project, we worked closely with neurodivergent children and their parents, discussing design and refining the app based on their feedback, to make sure we were properly supporting the target audience. We followed the steps of design thinking and the principles of universal design, considering the current struggles nonverbal neurodivergent children face, then ideating, prototyping, and testing.

Challenges we ran into

The main challenge we faced was generating the right cards efficiently and effectively. Our first attempt queried an LLM to output the twelve concepts most relevant to the conversation, then semantically searched for the most similar vocabulary cards within Cboard (app.cboard.io), our chosen word database, which also provided images. After testing, we found that this method was not effective, since the semantic search lacked the full context that the LLM had, leading to cards that were often poor options for what the child wanted to say.

Through trial and error, we settled on providing the full vocabulary list in the prompt for the LLM. Though we originally avoided this approach out of concern that the large vocabulary list would be too costly or overwhelming for the model, we found that Gemini 2.0 Flash could reliably select from the full list at low enough cost to make it viable. We also restructured the prompt around a “chain of thought” approach, where the model reasons about possible ways the child might respond and selects the cards that best help them convey their idea.

We also found our original UI was not user-friendly and often overstimulating in testing with neurodivergent children. After researching neurodivergent-friendly design, we opted for a simpler layout: a dyslexic-friendly font, clear dark borders around objects, a more neutral color scheme, and customization options to mute all sound and adjust text or card size.

Accomplishments that we're proud of

We are proud that we created a working app that can be actually used for conversation. We’re especially proud of hearing the excitement and feedback from parents, who seemed genuinely hopeful and grateful for something built for their kids. We’re also proud that we kept accessibility at the center of every design choice and didn’t have to cut corners anywhere. Most importantly, we’re proud that our product has the ability to improve the lives of nonverbal neurodivergent children.

What we learned

Through this project, we learned how important it is to design with users instead of only for them. We focused on making our project accessible and convenient for everyone, and learned how important real, live feedback can be.

What's next for Iris Speak

For the future, we plan to expand the vocabulary database used, support more languages, and work on making our app run quicker and more effectively. We plan to do more testing with neurodivergent children and continue reworking our product.