ClinicAi

Inspiration

Through my research internship at the CHU Sainte-Justine, I realized just how difficult it is for physicians to navigate the array of patient charts, lab reports and handwritten doctor notes. This makes it time consuming for clinicians to retrieve important information and accurately understand a patient's full picture, especially during handoffs and inter-department collaboration. This is what prompted us to create an AI-powered tool that makes it simple for clinicians to search through and access information in patient files, while getting essential answers necessary to their work. We are not replacing physicians, but rather assisting them in their jobs.

What it does

Our application first takes patient files (png and pfd) and processes them into tables that are easy to read and analyse. Clinicians can then use our chatbot feature to query the patient's chart and get relevant information (e.g. what abnormalities are found in the patient's tumor pathology report? or when was the patient's last blood test?). This allows physicians to efficiently retrieve information and validate their concerns.

How we built it

For the backend, we used python embedded with Gemini API to build the image processing tool, which takes in medical files (scanned png or pdf) and transforms it into a database on MongoDB, for each patient. Then, this data is fed to Gemini API to then serve as a basis for our chatbot functionality. Finally, we linked this to a html frontend.

Challenges we ran into

Trying to populate the MongoDB database using text extracted from images (i.e. pulling text from the scanned images) was a challenge, because we had never used it before.

Accomplishments that we're proud of

We are proud of the final image processing part of the backend, as it allows us to read a variety of different file formats and populates the MongoDB database in a logical way.

What we learned

We learned to efficiently use git as well as use new APIs such as MongoDB and Gemini API. We really enjoyed working with MongoDB as it is a flexible database that adjusts to different types of medical files. Also, everything in the image is structured and readable by the machine.

What's next for ClinicAi

We hope to develop a researcher section, that allows more specific requests for researchers. For instance, conducting statistical testing on data or populating a comprehensive table with multiple data elements for thousands of patients at once. Furthermore, we want to refine the search function by highlighting which documents the AI pulled each information from.