Good physician-patient communication (PPC) skills correlate with higher patient satisfaction, adherence to treatment plans [1], and better physical health outcomes like blood glucose and blood pressure control [2]. In addition to these patient benefits, physicians-in-training can develop better skills interacting with patients from studying those they see as expert - so-called 'master clinicians.' But what are the elements of communication, patient interaction, and empathy that make these particular physicians 'masters'? Certainly characterizing these traits will make it easier to educate other physicians in like manner. To date, however, the current 'gold standard' research instruments for studying the patient-physician communication involve manually watching full encounter videos and tagging particular conversation points. We would like to bring these types of important analyses into the digital age.

What it does

Diglossia provides physician clinical decision support tools, allows for physician self-assessment on their conversation with patients, and is an instrument for research all in one. It includes:

  1. Physician clinical decision support (CDS) tools:
    • Diagnosis aid: Listens to what the patient says and queries symptoms against a database of medical diagnoses to aid in generating a differential diagnosis.
    • Prompts for EHR tools: At relevant points in the encounter, physician is prompted to start an order set for a prescription, begin an order for a vaccine, or order blood work.
  2. Physician self-assessment:
    • Speech rate: Plots speech rates for patient and provider over the length of the encounter.
    • Amount of conversation space used: Plots distribution of time physician was speaking versus time patient was speaking.
    • Jargon detection: Detects potential medical or general jargon that may impede patient understanding and suggests replacement words for the physician to use.
  3. Instrument for research:
    • Feature extraction of 'good' PPC: quantitative features of PPC can be used to train a supervised learning model to predict such metrics as the results of patient satisfaction surveys.

How we built it

Front-end technologies:

  • Angular framework
  • HTML5
  • SCSS
  • Bootstrap

Back-end technologies:

  • Firebase
  • Firebase functions (NodeJS)

Third-party libraries:

  • Echarts (data visualizations)
  • Web Speech API (speech recognition)
  • Datamuse API (natural language statistics)
  • Apimedic API (diagnosis)

Diglossa was built using Angular with a firebase cloudstore database on the back-end as well as hosting via firebase. The app is broken up into discrete angular components which can be dynamically added to produce dashboards which provide meaningful insights into both the speech of health care providers and its intersection with patients. The app is a web app but is written to be as mobile friendly and portable as possible to enable doctors to use the platform either on tablets, phones, or laptop computers.

Challenges we ran into

One of the principal challenges which we faced is the lack of support for speaker diarization (distinguishing between speakers) in the Web Speech API. In the end, we determined that through noting pauses in speech, changes of speakers could be inferred yet this is an imperfect solution which would need a better long term solution eventually.

Additionally, our team was particularly challenged with the overall efficiency of the application. While we wanted to add as many features as possible, we quickly found that latency across the app began to reach exorbitant levels with the plethora of features we were adding continuously. As such, we made cuts for efficiency and simplicity to ensure the best user experience possible without the frustration of frequent freezing or lagging.

Accomplishments that we're proud of

We are very proud of the (though incomplete) product we managed to produce in just a couple days. The dashboard which shows time series data describing the doctor-patient interaction is something we truly feel can improve outcomes for patients across the board. The real time nature of these speech analytics is largely unprecedented in the medical field. This sense that we have somehow made a difference in the medical community in a substantive and unique way is very fulfilling.

What we learned

Doing this project taught us the complexities of PPC with so much available data yet not many contrived algorithms to fully exploit it to its full potential. We learned a lot about speech analysis and its intricacies in order to produce the best data visualizations possible.

What's next for Diglossia

We would like to improve speaker diarization, get our hands on some real clinical data to train real-world models, and undergo a UI overhaul.

Share this project: