When doctors provide consultations to patients, they often use medical jargons to elaborate on the prognosis, followed by the diagnosis and recommended procedures. Patients are mentally stressed by their medical condition and this affects their cognitive skills, hampering their ability to memorize the medical information given, recollect prescriptions, and remember the follow-up appointments. Even when patients try to jot down the details, they may omit several critical points amidst multi-tasking between note-taking and paying attention to the doctor. Some patient may resort to using voice recording. This however potentially sour the patient-doctor relationship as the recording at the hand of patients may be used as evidence against the doctor in the future.

Patients also find it difficult to navigate in a labyrinth-like hospital. It is common practice for doctors and nurses to tell patients which room or which floor to go to as for their next appointment or procedures. Although the rooms are labeled, and signage put up to ease navigation, the process of finding it when patients are unfamiliar with the place is still daunting.

What It Does

Improve the patients’ experience in the hospital in two areas: understanding verbose information given by medical professionals, and in-door navigation in the hospital.

How We Built It

We propose a cyber-physical interconnected system “Check” to connect the patients to the digitized hospital with the use of data and AWS. Check is a service that transcribes, organizes and translates information from doctor-patient consultations, making information concise and easy for the patient to understand. It also manages the patient’s scheduling for future appointments, sets reminders for prescriptions, and directs the patient to their next location* using data from the transcription, serving the patients as a personal concierge.

Technical Aspects

All of Check’s logical processing will be serverless; this helps to manage cost and Check can be run solely on managed services by Amazon Web Services (AWS). In this particular implementation of Check, high security with the use of Virtual Private Cloud (VPC) and private subnet, and fast data access speeds are prioritized. For this reason, the team has leveraged highly scalable and secure data storage strategy, namely: AWS ElastiCache for storing and lookup of session information and AWS DynamoDB for lookup of pairwise data.

AWS ElastiCache is used to cache hot session data that needs to be retrieved as quickly as possible, like the Session ID, for instance.

AWS DynamoDB is used to store warm data that is not as important, such as the patient secret key, the patient’s last 4 digits in their NRIC, transcription notes, and the mapping of directions for a source location to all destination locations.

Hence, regardless of the storage the data ends up in, they can still be accessed very quickly. As both services are managed services, AWS will automatically guarantee elasticity and durability for the mentioned storage services. Data at rest in the storage are encrypted.

As an alternative to secure infrastructure that is traditionally achieved with the AWS VPCs and AWS EC2s alongside subnets and security groups, and to save running costs right up to the cent, AWS Lambda is used.

The functions are secured by ensuring that the IAM role permissions assigned to them are necessary, and they are isolated within their own security groups and subnets to ensure that the AWS Lambda functions have no chance of interacting with one another unless permitted.

Another benefit of using AWS Lambda is that AWS does the scaling automatically for Check; which not only optimizes costs due to the per-request billing but also provides enough juice for the heaviest traffic.

Hence, by opting for AWS Lambda over AWS EC2 and by isolating each instance in a secure enclave via subnetting, Check provides efficient use of funding, high security and will scale to any size of current traffic.

Amazon Transcribe provides us with a high-performance, affordable speech-to-text transcription service. The speaker identification feature will assist to detect different speakers (doctor or patient) in the audio with high accuracy and confidence to produce intelligible who spoke when ​ transcriptions.

This allows us to extract the required information and pass it to Amazon Comprehend - a natural language processing (NLP) service that allows us to find insights and relationships in text.

Another advantage is that we could use custom vocabulary to improve the accuracy of speech recognition for singlish terms, medical domain-specific terminology, or names of individuals.

These give Amazon Transcribe more information about how to process speech in an audio clip which boosts the accuracy of the service. Amazon Transcribe is device agnostic, which means it works with any device that includes an on-device microphone such as phones, PCs, tablets, and IoT devices (microphones embedded into bed-side devices). We recommend the use of Echo Show for the end device.​ Therefore, this service is highly performant and reliable.

Amazon Comprehend analyses transcribed audio files from doctor-patient consultations to find insights and relationships in text. The service identifies and extracts key phrases, places, people and events in the transcription. This helps Check to organize information specific to locations (procedures to be followed when the patient is at a particular clinic or center), date and time (date and time of next appointment and important documents the patient might need to bring for his/her next visit), people (referrals to other doctors and medical personnel) and key phrases (prescription information and dietary restrictions).

Amazon Translate delivers rapid and high-quality translations, allowing us to serve patients with different linguistic needs. The transcribed and organized information can be viewed in a wide variety of languages that the service supports.

Accomplishments That We're Proud Of

Won first runner up & Viewers Choice Award in AWS Cloud Technology Hackathon 2018, to liaise​ with KK Hospital for implementation.

Built With

Share this project: