Real Time Symptom Logging

• 

  Screenshot of the user interface with a result

Inspiration

Healthcare can often be slowed down due to geographical barriers, which could be inconvenient or fatal. Patients in remote or underserved areas often travel long distances to seek medical help while some specialized healthcare is unavailable for certain regions. Furthermore, potential life-saving diagnoses can be halted by wait time or travel time. Most of the time, doctors need the same information on symptoms to make these diagnoses. Thus, having a remote way to log these symptoms for doctors to review over distances could not only be time-saving but also life-saving.

What it does

The Real-Time Symptom Logging takes audio input from the user as if the user were making a phone call. Then, it outputs a list of symptoms the user is having that could later be provided to a healthcare professional to speed up the process of providing a diagnosis and treatment for the user.

How we built it

All of our code is in Python. To begin with, we trained a natural language processor (BioBERT) to be better at NER for symptoms. We then adapted a library for STT use. Using the STT as the input for our fine-tuned NLP, we could tokenize the text. We then used this data alongside a comprehensive symptom database to produce an output on an interactive FLASK frontend.

Challenges we ran into

To begin with, BioBERT was not very accurate at specifically picking up symptoms. It is better for general medical use and not specifically diagnosis, so we decided to fine-tune it. This took a long time to work through. Many symptoms are single word only and easily identifiable, but multiple word symptoms ("broken neck") proved to a challenge. By using the scores from our NER, we intelligently made possible combinations of words and searched our symptom database for them as well.

Accomplishments that we're proud of

We are proud of our multiple word processing and fine-tuned NLP. Our front end is clean as well.

What we learned

We learned a lot about how natural language processors work and how their outputs can be used in many ways.

What's next for Real Time Symptom Logging

With a list of symptoms to provide healthcare professionals, the next step for our technology is to automatically pair patients with a list of professionals in the region to review their symptoms.

Built With

• ast
• biobert
• flask
• numpy
• pandas
• python
• re
• sounddevice
• speech-recognition
• torch
• transformers