When one of our members was shadowing a primary-care physician, he noticed that the quality of reports she made didn't reflect the quality of her medical care or her thought process - she had too many patients to be inefficient. We realized that, with the use of modern NLP algorithms, this is a solvable problem that can reduce the administrative workload doctors have to do, enabling them to spend more time with their patients on a deeper level.

What it does

How I built it

Frontend: The front end is a website built on HTML5, CSS, JavaScript, and React. We built it from scratch. We worked to create a simple website for an easy-to-understand software for the doctors.

Backend: The react frontend is supported by a nodejs/express backend that handles backend calls and the processing for the data. We built an Azure app with LUIS to leverage its NLP capabilities, as well as implementing the pre-processing required to use LUIS. The conversation between the doctor and the patient is classified into a SOAP(Subjective, Objective, Assessment, Plan) note, which also represents the intents we built into the Azure app. Using entities we created to represent medical information, we're able to group sentence fragments together that fall into one of the fields of SOAP. Then, we send the intents given the sentence fragments into a firebase database in a json format.

Challenges I ran into

Frontend: Since this was the first experience in web design for the 2 people creating the frontend, we encountered multiple problems. The two of us who worked on the frontend encountered problems ranging from creating popups to creating separate pages. In these problems, we had to fix imports, rewrite code, and research alternative solutions online.

Backend: Given the time constraints, we believed that it would be easiest to leverage Azure's power to build the NLP algorithm. We had trouble determining the efficacy of the LUIS app because some of the processing happens in the background, which is why we implemented some of the NLP functionality ourselves in addition to using LUIS. Moreover, LUIS isn't very good at detecting multiple intents given a single utterance, and so it took us some time to figure out how to pre-process the data we were sending to LUIS and then further interpreting the response.

Accomplishments that I'm proud of

Frontend: Learning HTML, CSS,JavaScript, and React.

Backend: Developing a working LUIS app Making a hack with the potential to help real people

What I learned

Frontend: Learning HTML, CSS,JavaScript, and React.

Backend: Learned how to use Azure, express, and nodejs.

What's next for Clinic Assistant

Frontend: Improve the aesthetics of the website with more CSS.
Convert the popups to separate pages.

Backend: Improving the NLP algorithm.

Share this project: