Personal Virtual Nurse Assistant

Inspiration

The inspiration for the Personal Virtual Nurse Assistant (VPNA) stems from the growing need for personalized, accessible, and continuous healthcare support. Many patients struggle with managing their medical history, prescriptions, and test results while lacking immediate access to healthcare professionals for routine queries. Our goal is to create a virtual assistant that acts as a personal nurse, offering tailored advice, maintaining a memory of patient interactions, and analyzing medical documents to provide accurate and context-aware responses.

What it does

The VPNA is an AI-powered chatbot designed to:

1. Onboard Patients : Collects medical history, allergies, medications, and lifestyle habits through an interactive questionnaire or form.
2. Store Memories : Saves all patient interactions in a vector database as embeddings for future reference.
3. Answer Queries : Uses a fine-tuned language model (unsloth/DeepSeek-R1-Distill-Llama-8B) to provide medically accurate responses.
4. Process Documents : Allows users to upload PDFs of medical reports or prescriptions, which are stored as embeddings and used to enhance responses.
5. Dynamic Context Awareness : Summarizes past interactions and appends them to each query to maintain continuity and relevance. For example: A diabetic patient can ask, "What should my blood sugar levels be?" and receive a response based on their uploaded lab results and medical history. A user can upload a prescription PDF and ask, "Can I take this medicine with alcohol?" to get a tailored answer.

How we built it

We built the VPNA using the following components:-

1. Model Fine-Tuning : Fine-tuned the unsloth/DeepSeek-R1-Distill-Llama-8B model on the FreedomIntelligence/medical-o1-reasoning-SFT dataset for domain-specific accuracy.
2. Vector Database : Used ChromaDB to store patient interactions and document embeddings.
3. Frontend : Open WebUI Backend : Built with converting the fine tuned safetensors to gguf and to deploy on Ollama to handle API calls to the fine-tuned model and vector database.
4. Document Processing : Leveraged libraries like PyPDF2 or pdfplumber to extract text from uploaded PDFs and generate embeddings using sentence-transformers.

Challenges we ran into:

1. Data Privacy Compliance :

• Ensuring compliance with healthcare data regulations was a significant challenge, especially given the sensitive nature of medical information.

• In Canada , we adhered to the Personal Information Protection and Electronic Documents Act (PIPEDA) , which governs how private-sector organizations collect, use, and disclose personal information.

• In the USA , we ensured compliance with the Health Insurance Portability and Accountability Act (HIPAA) , which sets standards for protecting sensitive patient health information.

• Both frameworks require strict measures to safeguard patient data, including encryption, access controls, and regular audits.

1. Memory Management : Balancing the amount of context appended to each query to avoid overwhelming the model while maintaining relevance was a technical hurdle.
2. Fine-Tuning Complexity : Achieving high accuracy on medical datasets required extensive experimentation and validation to ensure the model provided reliable and clinically sound advice.
3. Scalability : Designing the system to handle large volumes of patient data and concurrent users while maintaining low latency was challenging.

Accomplishments that we're proud of

What we learned

What's next for Personal Virtual Nurse Assistant

Built With

• deepseek
• huggingface
• torch
• unsloth
• wanb