Healthcare Helper

Register as a Patient
Register as a Doctor
Patient Chat
Login Page
Google AI Breadboard

Inspiration

Multiple sources show that the healthcare industry is critically understaffed, leading to burnout among healthcare professionals, long patient wait times, and ultimately, poorer health outcomes. We thought: is there any way that AI could help streamline the healthcare process to address this problem? We have all had experiences where doctors spend a lot of time asking questions, many of which could have been addressed before the visit.

What it does

We created the Healthcare Helper, which streamlines this process by using patient data and “the reason for visit” to ask the patient personalized questions to gain more relevant information about the patient's health. The Healthcare Helper then collects all this information and reports to the doctor: an exact transcript of the conversation, a summary of the most important points, potential diagnoses, and a list of any concerns that may warrant further investigation. The doctor can view this before a patient visit and gain more information to better and more efficiently help the patient.

How we built it

We leveraged Google AI’s Breadboard to create a system consisting of three Gemini 1.5 Flash AI Models that interact with one another, each one tailored to a specific role. Upon making an API call that inputs the patient’s medical information to the Breadboard, one of the AIs, known as the Planner AI, uses a “for” loop to facilitate a conversation between the Healthcare Helper AI and the patient. The Planner takes the patient’s medical information into account to form guiding topics that the Helper can use to formulate questions. After the Planner has collected sufficient information about the patient visit, the conversation is output to a Summarizer AI that outputs the conversation and summary. This information is then fed back to the website and stored as a patient record, which only doctors may access. We utilized Python, React, Flask, and SQLite to create a web application. We called the AI Breadboard and Cartesia as APIs.

Challenges we ran into

Our primary challenges were integrating the Breadboard project to our frontend website, and creating the website from scratch. Setting up the website to be functional was a lengthy process on its own, and we spent time designing web pages and databases to fill out the site as well. We also had to dedicate some time understanding the Google AI Breadboard documentation in order to develop our own project and use API calls on it, since it was a new technology we had never encountered previously. Additionally, since it is a very new technology, the documentation is not very thorough, so we had to figure a lot out on our own.

We also encountered challenges when controlling the output of the AI. We were getting repeat questions, and inconsistent formatting, which led to poor user experience and trouble parsing the output. We were able to solve this with a lot of trial and error, and some very specific prompt engineering

Accomplishments that we're proud of

We were able to integrate results from a bunch of unique sources. Combining these separate elements into one efficient and sleek website proved to be very challenging. Attempting to maximize the efficiency of the AI Breadboard was very rewarding for us as we were able to achieve great results because of it. We were very happy with the questions it was asking, its interpretation of the conversation, and the consistency of the output format.

What we learned

We had never used the Google AI Breadboard tool before, so we were able to experiment and learn about the interaction of multiple AIs. As the world shifts more towards AI, understanding how they interact with each other will prove invaluable for our future in the industry.

We learned a lot about the healthcare industry, and the extent of the challenges they are facing. This project and our research has definitely inspired us to learn more about how we, as engineers, can help with their work.

What's next for Healthcare Helper

An important step that we did not have time to take was fine-tuning the Gemini AI models so it would be much more accurate for healthcare and diagnosing. This would greatly improve the quality of the interview questions asked to the patient, and the interpretation given to the doctor. The next step for Healthcare Helper is becoming a full-fledged healthcare platform, where patients can view their diagnoses or medication, where they can request appointments, or send direct messages to their doctors, and where doctors can easily manage and interact with their patients.