PreScribe

Diagram of Application

Inspiration

Our healthcare workers are overworked. Nurses and doctors spend around a daily median of 4.5 hours on EHR notes. This type of draining work would affect the quality of anyone's work. It's not just doctors that are affected. A frequent complaint on the part of patients is that they don't feel like their being listened to when their doctor is forced to stay in front of a screen.

As such we've seen an explosion of AI scribes which try to relieve this problem. But one avenue that hasn't been fully exploited is using this technology to aid doctors in maximizing patient safety.

We also know the effectiveness of checklists. This is one of the reasons why the aviation industry is so safe because of their emphasis on them. But adding more work on the part of the doctor to keep up with the latest checklists would only worsen the bureaucracy paperwork problem which is the biggest cause for our healthcare workers wanting to leave within two years.

We think we can use state-of-the-art Large Language Models to make AI agents that work as a copilot to help reduce medication and diagnostic errors on top of this great work-reducing transcription technology. The agents can keep up to date on the latest checklists and help reduce burnout from paperwork while healthcare workers can focus on care for the patient.

What it does

Our program, PreScribe, records a conversation between the doctor and the patient and transcribes it. We've specifically done this by having the best text-to-speech model by OpenAI running to continuously update the EHR with the patient-doctor conversation on your phone for ease of use.

We've specifically modified the model to increase its accuracy with medical jargon. We have another process where an LLM organizes this information with patient/doctor labels to the best of its abilities. This works on the paperwork/EHR inputting problem so that the doctor can focus on the patient to maximize the quality of the care.

We then can maximize the patient safety side of things by having multiple agentic LLM processes running against the transcript to work as a copilot to the doctor during the conversation.

We have one process that continuously checks whether the doctor is prescribing the patient a drug that would require a checklist. That checklist is made from reputable sources like .gov site MedlinePlus. This keeps the doctor up to on the latest developments without adding any administrative overhead!

We also have another agentic process that takes the real-time transcript and updates the checklist that's been queried as there's more information from the patient and doctor. This keeps both the patient and doctor visually informed of what is going on as if you had a third person in the conversation keeping track of what's going on.

With PreScribe we can alleviate the stress of our health care workers while also working on pressing issues of both medication errors and diagnostic errors.

How we built it

We used React Native and Expo to make the mobile app which records the conversation and sends it to our FastAPI server. Then, our server calls OpenAI's transcription API to convert the audio to text. Next, the server uses LLMs to detect whether the transcription satisfies the constraints on the checklist. The changes in the transcription and the checklist are uploaded to a Firebase database which the front-end dashboard listens to for changes.

Challenges we ran into

We had trouble recording audio on the React Native app due to multiple race conditions. We also had race conditions on the frontend when trying to stream text to the user interface.

Accomplishments that we're proud of

This was my first hackathon and I learned a lot about software development, such as APIs, Git, and collaborative coding. We are proud of contributing to patient safety and demonstrating a possible path to reduce diagnostic mistakes.

What we learned

We learned more about how to use React Native, Modal, and LLM agent orchestration.

What's next for PreScribe

We can expand PreScribe to have a visual component that deals with surgical mistakes or helps to check off more requirements from the checklist. We think that when we have cheaper multimodal LLMS for video this could be very useful in surgery as well.