Tune to Talk

Inspiration

⅓ of stroke survivors suffer from aphasia—a speech disorder resulting from damage to the language centers of the brain. In fact, according to the American National Aphasia Association more people have aphasia than Parkinson’s or cerebral palsy. Yet the general population is often unfamiliar with this condition and the devastating effects it can have on a person's communication.

In cases of non fluent aphasia, patients struggle to express their thoughts. Though their intelligence is preserved, they may stutter or produce very few utterances. This can lead to frustration and the feeling of being trapped in one’s mind. However, many aphasia patients have seen improvements in communication through music therapy! Because speech and song are not processed by the same brain regions, aphasia patients can often sing more than they can say.

We ourselves became acquainted with this research thanks to our fellow hacker, Sydney, and her volunteer work with the Aphasia Institute of Toronto. Thus, we set out creating Tune to Talk—a game designed to help aphasia patients practice their speech through song. Think of a Quizlet for aphasia that can be used to make treatment more accessible and fun.

What it does

Tune to Talk is an interactive PC game that trains aphasia patients on useful, everyday phrases set to familiar tunes. Inspired by exercises used in melodic intonation therapy, users repeatedly sing phrases to work towards speaking those phrases.

Users can choose from a selection of decks organized by theme: greetings, needs, gratitude, feelings, and health. In practice mode, users are prompted to repeatedly sing each phrase in the deck set to the tune of a familiar song such as “The Addams Family.” Each card features the written phrase as well as a video of a human singing it.

In test mode, users can attempt speaking the same phrases they sang in practice mode. If the attempt is accurate they can advance to the next card and if not they are prompted to try again.

How we built it

The front-end team developed a prototype of the game using Figma and Adobe Illustrator. We also recorded our own audiovisual clips to be displayed on each card of the deck, and the back-end team used Godot game engine as well as Visual Studio Code to develop the final game.

Challenges we ran into

• Keeping in mind our target audience (stroke survivors and likely elderly), we had to consider simpler controls and designs that would not be overstimulating/be accessible
• Coming up with short, useful phrases that could fit within known melodies
• Getting an accurate reading of the text to speech
• Converting between media file types—specifically MOV to OVG

Accomplishments that we're proud of

• Our game is designed to improve the lives of a disadvantaged group
• Staying awake and working well together as a team, even though we just met

What we learned

• Some of us did not even know that aphasia existed. It is more common than you think and making this game helped us gain a better understanding of the condition itself and how it can be treated!

What's next for Tune to Talk

In future iterations of Tune to Talk, we hope to expand the decks and allow for customization options that would make the phrases more relevant to the individual user. Ideally, the game would be used as a supplement to in-person speech therapy. The content could be modified by SLPs who could use the game as “homework” for their patients.

Built With

• adobe-illustrator
• figma
• godot
• visual-studio