HEALTH Line

An offline dial-in system that recognises the users spoken language at utilises it at the point of contact with healthcare services using Conversational AI powered by Reinforcement Learning.

Comment

Inspiration

Dr Yin Lim, our team member has shared her experiences working with refugees and their families in accessing healthcare and identified that the poor engagement with healthcare services is largely due to the language barrier. Refugees and asylum seekers who do not speak english require significant language support at the every point of contact with healthcare services, and this support is often via charity organizations or friends, which is a significant administration task and barrier to access healthcare. Occasions where non native english speakers arrived to the clinic, people were not able to help because the spoken language could not be identified.

What it does

  1. Identifies the language of the patient on the hotline and subsequently interacts with the patient in their own native language to book appointments and liase with healthcare services.

  2. The user can book a healthcare appointment by talking with a conversational AI. The chatbot first recognizes the language and book a suitable appointment by analysing the symptoms. The systems can collect data and alert the healthcare providers of the patient's language needs in order for efficient delivery of healthcare and saves cost of admin time and clinician time, as well as interpreter services.

How we built it

  1. A portion of user audio while speaking is stored in to the AWS S3. which then used to detect the language using AWS transcribe.

  2. After language detection the audio stream gets connected to a conversational AI/chatbot.

  3. The AI-agent then walks through the booking procedure. It understand the nuance, utterance, intent of the user and using the pretrained AI model for disease detection from the given symptoms, it identify the specialist, book the ticket accordingly. It also sends the symptoms to the doctor for reference.

  4. Since the bot uses Reinforcement learning, it gets fine tuned as the conversation progress.

Challenges we ran into

  1. Technicalities : Training the doctor prediction from symptom taken much more time than we expected, conversational AI development was challenging in a short time frame.

  2. Unable to use online /graphical tools in our design due to the understanding that underpriveliged and underserved groups often do not own a smartphone or have access to the internet.

  3. Patient safety: Systems had to be built in place to ensure emergency medical conditions are addressed and patients are redirected appropriately to an emergency department for any high risk symptoms.

Accomplishments that we're proud of

  1. Developed a proof of concept

  2. Team members from different countries and time zones came together to complete this project.

  3. We were in agreement that this idea was important and relevant and had a similar vision for what the technology can achieve.

Built With

Share this project:

Updates