Problem Statement: AphasiaSG:
How can we enable people with Aphasia (PWA) in Singapore to return to meaningful engagement or employment in the community?
Background information about PWA (Extracted from Aphasia.sg):
Aphasia is an acquired communication disorder impairing part or all of a person’s use of language, which usually occurs after parts of the brain controlling language are damaged (eg after a stroke, head trauma, brain tumour), impairing a person’s ability to speak, write and/or understand words.
A person with Aphasia usually has troubles communicating the words to say in a way that makes sense to people. They may also have problems spelling words and understanding what was read.
93% of PWA suffer from psychological distress and depression as a result of loss of independence and social isolation within the first 6 months of their stroke, and has greater negative impact on a person’s quality of life compared to cancer or dementia.
Alphasia is a smart phone application that can allow a PWA to be more forward in communicating with people in a sense whereby they know that they have an application to fall back on should there be any trouble thinking up of a way to communicate.
Alphasia aims to aid PWA in becoming more dominant in terms of taking charge and initiating conversations with people. The main components are: About, Icons, Draw, Work and Speech to Image Translate.
About Allows PWA to have an introduction for the other party who they are talking to. This allows the person to understand more about the PWA, as well as how to best communicate with him.
Icons Provides the PWA to fall back to drawings to articulate what he/she wants to talk about. Icons can be used as an aid to elaborate / convey what PWA is trying to say, and words on top help associate the image with the word for the PWA or for the other party.
Draw Provides the PWA with a canvas to freehand draw and articulate his/her thoughts through drawing. Interface kept as clean as possible to minimize complications, comes only with eraser and clear commands.
Work Allows companies to work with PWA and create an interface for them to have a tool to work along with, should there be any potential problem in communicating. Example job: Barista, where he needs to get the order from the customer. By showing the customer the application, PWA can know what the customer wants to order in a pictorial form and have a better working experience instead of worrying about mixing up/not understanding orders.
Speech to Image translator Intention: For the PWA to understand what the other party is saying through visual aids. It works by first translating the speech to text, then subsequently followed by a text-to-image association. Current Prototype: Some words associated with the images inside the application, Can show up to 3 images, if there are any images. Some of the words in the dictionary are: sunny, cloudy, pizza, sushi, coffee, cockroach, photo, cat, dog and paint. Try saying this sentence: "The cat ate the pizza, and the cockroach" :)
How I built it
I built it using the MIT app inventor, which is a interface that allows for building quick prototypes of solutions. The idea can be brought over to Android Studio which allows for better scaling as well as designing.
Challenges I ran into
I was initially building it using react-native, but i ran into problems with some of the library dependencies. Therefore i had to roll back to another familiar application building program called MIT-App inventor. I also had some trouble when working with the limited interface for the speech to image translator as MIT app inventor is rather limited.
What's next for Alphasia - For People with Aphasia
The next step for Alphasia is for it to be ported over to Android Studio, as well as to fully develop the Speech to Image translation. There are some APIs that allow for the deciphering of the entire sentence, which allows us to place images in certain ways that paint a better picture for PWAs to visualise the image better. The goal of the Speech to Image translation is to allow any person to be able to decipher the image translated to what a normal person can understand.
If the solution i proposed is finetuned and the work portion of it managed well, i am sure that it can be provided to PWA as a cost-free solution, being integral in any PWAs in terms of employability as well as their engagement in the current society.
Demo Link (Youtube): https://youtu.be/BPGAyolANgM