Medical/Insurance File Processor and Assistant

Inspiration

The IBM challenge really inspired us since we are a team composed of a software engineering major, a double major in neuroscience and computer science, and a biomedical engineering major. Due to our cross display background we agreed that this would be a fun challenge to accept.

What it does

The program takes a pdf input file from the user, converts it to text, lets the ai read and process it, and then takes further text prompt questions from the server and answers their questions.

How we built it

We used python, streamlit, langchain and other tools. We first made a pdf to text converter, then we had the AI read it line by line. The text of the pdf was stored as a string variable. The AI then used info from this text to answers question's the user had. It was primarily based on a streamlit base code AI chat bot that uses a search engine for info, but we modified it to use info from a pdf.

Challenges we ran into

We were originally going to separately work on front end and back end portions of the program, but we had trouble meshing the two, as there were different languages used. Furthermore, the AI chat bot that we made using streamlit was very sensitive to additional code, and was not able to be meshed into HTML and CSS made code. We also struggled with finding the portion of the code to replace the part where the AI uses information from the search engine, and replace it with a pdf. Also, the original streamlit code was written with OpenAI in mind, but we couldn't get access to it's API key, so we had to use Cohere, which meant we had to update, and change lots of syntax surrounding the use of APIs in the code.

Accomplishments that we're proud of

Getting the AI to read the pdf file! Getting Cohere to work! Getting reasonable answers from the AI instead of just errors, or assertion errors for files larger then 2 sentences.

What we learned

Anna learned how to program in HTML, CSS, and JavaScript and she also learned the basics to AI and user design. Mo learned how to use parcel-cache to see my code on the local server. She also learned how to link an HTML file with JavaScript file, and how to use OpenAI to read a file. Cherry learned hot to use streamlit, as well as how to work with AIs in general, she also learned more about python beyond just basic computation commands, as well as gained a deeper understanding of python syntax, and how python works, which allowed her to make educated guesses as time went on about how to troubleshoot when encountering errors.

What's next for Medical/Insurance File Processor and Assistant

Get the program to accepts all sorts of files, as well as larger file sizes. Get it to display one singular answer instead of multiple responses the user has to click through, as well as masking errors, so that the user, should they encounter an error, is met with just a simple error message, instead of the full berth of the error message that pops up the code layers. Also, the API key should be built in, instead of having the user enter the API key.

Built With

• cohere
• css
• html5
• javascript
• langchain
• openai
• parcel-cache
• pypdf2
• python
• streamlit