Inspiration
Changing the World of Medicine
It's tough to find a doctor!
I'm sure we've all been there: some minor health problem that we maybe want to get looked at, but we're not sure who to go to. Maybe we're in a new place, maybe we just don't want to deal with calling around everywhere to see who takes our insurance, and maybe we're just tired of dealing with the whole process all together.
Sometimes, we just want to know what might be causing our minor medical issue!
Another situation we've all been in: a mark appeared on your skin and you Google'd what it might be and the web told you that you have skin cancer, when it was really just a scratch. This is so frustrating! It is almost impossible to find information to help us at least get a hint of what our day to day minor medical issues might be!
The internet is just TOO huge!
Finding doctors online is the WORST! there are just so many different variables and searching on the internet just never seems to yield any results we want! Maybe I don't even know what kind of doctor I need for my issue, or maybe I'm being mislead online to another doctor that might not be right for my requirements.
These reasons are all the inspiration for HealtChat! An AI-powered Medical Search System!
What it does
HealthChat uses AI to answer patient questions to help them better find a doctor that suites their needs!
Gone are the days of endless searching when you need the services of a medical professional. With HealthChat you can ask questions about your medical issue and be provided with doctors in your area that match your criteria. Using the Bing GPT4 Engine, HealthChat is able to search the internet for you, using the most reliable sources first.
2 Main Features:
Chatting
Ask the HealthChat Engine a medical quesiton, and it will search the internet to gather data to provide you the best possible response. Users can use HealthChat to better understand the issue they are having to get the care they deserve Faster
Physician Matching
Ask the HealthChat Engine to match you with a physician and you will be give a form to help it better understand your needs. Using its powerful AI Engine, HealthChat will gather information on physician that specialize in your direct need without you needing to do any research. Users can simply describe their issue and be suggested physicians nearby, even checking that they take their insurance!
How we built it
HealthChat is a complete, polished Next.js application.
Tech Stack Next.js React/TypeScript MaterialUI 5
HealthChat leverages the newest, most powerful web technologies in order to pull off this daunting task all while looking sleek!
HealthChat runs a NodeJS wrapper for Microsoft's Bing GPT4 engine.
By utilizing open-source software from the community, I was able to use Prompt Engineering to come up with 2 prompts: one for medical questioning, one for connecting to doctors. This makes it so our Bing GPT4 Engine can return consistent, mostly accurate JSON data no matter what we throw at it. By utilizing this engine for data gathering, we do not need to worry about the responses not following a consistent format: Bing GPT4 will search the web and use the data to give back a JSON object every time.
The open source Bing GPT4 wrapper that helps power HealthChat can be found here. A community made hack perfectly fits in with the theme of a hackathon!
Challenges we ran into
A major challenge was finding an engine that can run this thing. I test drove many differnet solutions:
- OpenAI's API
- Google Bard's Unofficial API
- Microsoft Cognitive Services
- IBM WatsonX Assistant
And a few other community made Open Source solutions. Unfortunately, none of these APIs are able to search the internet, until I stumbled upon the Bing GPT4 wrapper. Now, I was able to utilize Microsoft's most powerful AI agent yet, with the next challenge engineering a prompt that would force the same structured reply from the AI assistant each time. I settled on JSON, and the two prompts I engineered are:
Medical Problem:
{ MEDICAL_ISSUE }
Identify which specialty would be best to treat it
Find doctors in or near: { CITY , STATE } that match that specialty and accept the insurance: { INSURANCE }.
Please do not respond with any symbols, only plaintext. If you must link a site, only respond with the link's url.
Fill in this JSON (doctors is a list of doctors, an example doctor object is shown) as your response, and respond with only the filled out JSON form below. Nothing else. Try to fill as many doctors and as much info in the JSON as you can:
{
"specialty": <answer>
"doctors": [
{
"doctorname": "<name>",
"address": "<address>",
"phone_number": "<phone number>",
"website": "<website if exists, else leave string empty>",
"board_certified": "<true or false, if board certification is not an thing for this kind of speciality, just type N/A.>",
"rating": <the doctors review out of 5 stars, if you can find it. Please return the integer representing the rating. Otherwise, return -1>"
}
]
}
If you do not think the medical problem stated is a medical problem, return the below JSON instead. Remember, it is ok to return the above JSON with no doctors, this should only be used when you have tried everything and cannot determine this to be a medical issue.
{
"error": "Unknown Request"
}
and
Medical Problem:
{ MEDICAL_ISSUE }
Search the Mayo Clinic and other repuatable medical sources to try to come up with what might be causing this symptom(s)/problem(s).
Please do not respond with any symbols or markdown/special characters, only plaintext. If you must link a site, only respond with the link's url. The question may not fill every section of the below JSON and that is ok, sometimes a general description is all that is needed.
Fill in this JSON (possibles is a list of possible issues that might be causing it, an example possible object is shown) as your response, and respond with only the filled out JSON form below. Nothing else. Try to fill as many possibles and as much info in the JSON as you can:
{
"general_desc": "<give a short overview of findings. Keep it limited please>"
"possibles": [
{
"possible_name": "<name of what might be causing it>"
"source": "<link your source>"
"desc": "<give a short description about what this is. Keep it limited please>"
}
]
}
If you do not think the medical problem stated is a medical problem, return the below JSON instead. Remember, it is ok to return the above JSON with no possibles or empty, this should only be used when you have tried everything and cannot determine this to be a medical issue.
{
"error": "Unknown Request"
}
These two prompts will always force bing to return us a JSON file that we can parse in TypeScript. The only remaining issue here is speed: Bing takes a while to search the web and gather data. Since it's a closed source engine there's not too much I can do about that at the moment, but some of this bottleneck also is on my end and the wrapper's end, both of which I could contribute to to make better!
Creating a chat interface:
This was a big one. Creating an interactive chat interface was tough! I needed to create a lot of React hooks and effects to mimick that text message-like feel that modern AI systems can achieve. Using React really helped here, as I was able to build each bubble as a component and manipulate them where I saw fit. MaterialUI was also a huge time saver here, as it meant I didn't need to spend a ton of time creating smaller components like boxes and styled strings, instead I could focus on making the site shine!
Hosting
The third biggest issue of this project was hosting! It's the only I don't have figured out yet (more on that below). Basically, Microsoft doesn't like when you try to access Bing GPT4 from places it deems "unsafe." It forces a reCAPTCHA which causes a websocket error when you host the site. I've been scraping through the Microsoft docs to see how to get around this, but this is the only large unsolved issue of the project! Originally I thought it was my cookie value expiring, but it seems that is ok!
Accomplishments that we're proud of
HealthChat is Completed!
That's right! In 24 hours I was able to single handedly put together this beast of an application and get it to a functional first version. I'm extremely proud of this as I had thought I would never get even close to here, so this is pretty crazy to me! I would love to add more features down the line (more on that below). It's also supported across most mobile devices, and has been tested against all Chromium development viewports!
HealthChat has an interactive, modern feeling UI
I feel like for most of my CS career I've been a backend engineer with some frontend work here or there. Creating a user interface as friendly and intuitive as HealthChat's has been an incrediby accomplishment for me. I actually spent some time before writing this submission just playing with the Chatting feature because of how cool it is and natural it feels!
HealthChat could actually make a huge difference in the lives of many
Seeing the use cases for HealthChat has made me realize just how many lives this could actually help, and it feels really incredible to say I developed a piece of software that could change the world!
What we learned
I've learned that AI isn't really all that scary. I've worked with it in class before, yes, but absolutely nothing to this scale. Using modern AI tools and incorporating them into my project was an incredibly rewarding experience, and one I will definitely carry with me forever.
I also learned a TON about UI design, and pretty much taught myself as much as I could about Front End Development in these 24 hours. It was a crazy rush! I've never coded nonstop for nearly an entire day before so this was truly something.
I learned a lot about the interworkings of Nextjs. I've worked with it before but this time I actually got to discover routing and how it handles requests, it's quite different to Express and FastAPI!
And finally, I learned what it means to develop software with your target audience in mind. I had to constantly remind myself that no matter how difficult, creating a seamless experience and streamlining the search for doctors is something that many people COULD actually benefit from. Stopping every so often and taking a look back at my original idea and customer use cases definitely helped a ton with this!
What's next for HealthChat
Hosting
I'm sure there's a way to host this thing. The problem as I said above is that Microsoft is blocking my requests out causing websocket errors. This is definitely not cool, and I'm totally going to do more research into how to get by this after the hackathon concludes.
Optimization
I definitely want to contribute to that Bing wrapper I've been talking about. That way, I can better optimize my Next routes to significantly reduce the wait time.
Documentation
Since I wrote this so fast, my code could use a little cleanup! I would love to go back through and get some proper docustrings and comments setup throughout, I can already think of a few things that I can probably throw out as well that are still in the codebase.
Physician Portal
An idea I had for this was instead of just giving contact information, I could also have a system for Physicians to sign up and communicate directly in the chat window with the patient, having actual medical advice given outside of the AI processes, with the AI acting as an agent that connects the patient to a physician.
Thank you so much for this opportunity. This was incredibly fun! Here's a link to the hosted service, but remember the AI agent will not work when hosted. You can clone the repo and run it and try it out as well! https://symphonious-fenglisu-9ce131.netlify.app/ https://github.com/anderm18/HealthChat/
Built With
- bing
- gpt4
- materialui
- next.js
- node.js
- react
- typescript
Log in or sign up for Devpost to join the conversation.