Openvino Chat with PDF

Inspiration

What it does

This is a chat bot which will answer your queries based on the set of pdf documents. It can also summarize the contents of PDF.

How we built it

We have build it using Openvino, Streamlit and langchain. and running as container in Openshift Cluster on Intel CPU.

Challenges we ran into

We have faced challenges with model hallucination.

Accomplishments that we're proud of

We are able to run RAG application on Intel CPU using small parameter model. which is LLama 3.2 3b

What we learned

We learned how to control hallucination and inference Model on cpu using openvino.

What's next for Openvino Chat with PDF

We will add multiple document type and enable user login and persist data using collections.

Built With

• chromadb
• huggingface
• langchain
• llama
• openvino
• python
• streamlit