PatientPal

Chat AI feature for patient real-time questions with multi-lingual support.
Sample empty Chat AI feature
Medications unexplained
Lab results input
Lab results no input chosen
Medication summary
Lab results summary with English
Medication summary with multi-lingual support
Lab results summary with Spanish

Inspiration

Reading medical reports is never fun. They're long, full of jargon, and filled with news that range from a checkup request to life-changing. And so we thought: what if it didn't have to be this way? What if everyone with a click of a button understood exactly what your doctor wanted to tell you?

What it does

PatientPal is designed to be an anonymous tool where anyone with a confusing medical report can securely upload it and receive a detailed summary of their medical report in an easy-to-read format. Our application can add its interpretation and draw conclusions based on the user's lab results. Lastly, PatientPal also translates the final text from English to other languages so that no one will be harmed due to not understanding that they have a tumor or diabetes in their blood work.

How we built it

This application is powered largely by Google's Healthcare, Gemini, and Translation APIs. Our frontend was created with React.JS and our backend was built with Python using FastAPI. The Healthcare API was used to extract medical information from text, formulating new conclusions as it categorizes the information it finds. Gemini served as a chat client and a powerful tool to summarize the massive data stream we received from the Healthcare API. Google's Translation API was used at the end of this pipeline to translate the Gemini summary into Spanish.

Challenges we ran into

Our main challenge was implementing all these features into one application and building a UI for users to interface with the API. As new users of the Google suite of APIs, we found it challenging to find a foothold in our API development process. However, we persevered and built a functional proof-of-concept API with just enough time to build a mock UI.

Accomplishments that we're proud of

We managed to synthesize multiple APIs into one coherent application despite the difficulties and time constraints. We never used these tools before and the achievements we accomplished in the face of that to build something new is something we are proud of.

What we learned

Our team learned the tools that were described before, but we also learned how to utilize them. Things such as Google Service Account authentication, FHIR data, batch processing, and interfacing with Gemini context windows among many other tools and techniques would have taken a long time otherwise to get exposure to.

What's next for PatientPal

We plan to finish our exploration into this project to make a finished project, but we also believe many others can learn from this project which is why we plan on leaving this project open-source.

Built With

Submitted to

LA Hacks 2024

Created by

I worked on the backend for ntegrating GeminiAPI, translation, and Speech to text features.

Tanvi Guttula
In frontend development, I utilized a combination of React.js, JavaScript, and CSS to craft an engaging user interface. Leveraging React.js components like headers, sidebars, and chatboxes, I ensured seamless user interaction. With JavaScript, I enhanced functionality and responsiveness, creating visually appealing designs that prioritize user experience.

Abegail Palad
I worked on connecting the frontend with the functionality in the backend. I used React.js and Axios to make calls to the backend APIs written by my teammates.

Sarah Singh
Andrew Lao