Empowering Doctors with Real-Time Insights from Your Fitness Wearable 💪👩‍⚕️

💡 Inspiration

Doctors often find themselves overwhelmed with too much data - especially now with wearable health platforms. The critical insights that could transform patient care frequently remain buried. MosaicHealth AI helps sift through this mountain of data to highlight the most important information that truly matters.

🧙‍♂️ What it Does

Imagine a world where your fitness wearable isn't just a passive observer but an active participant in your health journey. MosaicHealth AI is that vision brought to life. As doctors and patients engage in conversation, our system listens in, displays key health insights from the patient's wearable data in real-time. This isn't just data collection; it's data revelation, assisting doctors in crafting medical reports with unparalleled precision and insight.

🤖 Intel Developer Cloud -

Prediction Guard APIs - Secure, compliant LLM
Intel Distribution of Modin - improved inference speed for data analysis on wearable data using Modin-powered pandas
Phi-2 finetuning using IDC and medical dataset - finetuned Phi-2 using one of HF's biggest medical datasets
Submitted to HF leaderboard - submitted Intel optimized model on HF leaderboard
Int8 Quantization of Diaalo-GPT-large using IPEX - (experimentation) faster inference speed after int8 quantization

HuggingFace submission - https://huggingface.co/sohampatil/msphi2-medical

🏗️ How We Built It

• Harnessing Wearable Data: Using Intel Modin, we analyze fitness wearable data from devices like your Apple Watch or Peloton app, extracting relevant information about your condition for the doctor's review.
• Embracing AI with Care: Recognizing doctors' apprehension towards AI, we integrate Prediction Guard's HIPAA-compliant LLMs, marrying compliance with the forefront of technology.
• Fine-Tuning with Precision: We fine-tuned Microsoft Phi-2 using the MedMCQA dataset from openlifesciences, adding knowledge from medical entrance exam questions for enhanced contextual understanding.
• Real-Time Data Extraction: Utilizing on-device WebAPIs, we actively listen to patient-doctor conversations, pinpointing critical health insights and amalgamating them with Apple Watch data, all processed through Intel's Modin library and LLM APIs.
• Intelligent Report Generation: Leveraging PredictionGuard, we amalgamate narrative and numerical health data to meticulously craft comprehensive medical reports.

🧗‍♂️ Challenges We Ran Into

Navigating the labyrinth of medical privacy and data security, we chose Prediction Guard to shield our AI from the pitfalls of non-compliance and the specter of hallucinations, and to make doctors feel safer.
Bridging the gap between spoken words and LLMs was a formidable challenge + the process of learning model fine-tuning.

🤯 What We Learned

Our journey with MosaicHealth AI has been a profound learning experience, diving deep into the realms of Intel's Developer console, mastering the nuances of JavaScript, and unraveling the complexities of AI fine-tuning. We've harnessed the power of Large Language Models (LLMs) through advanced APIs, gaining invaluable insights and skills along the way.

🏆 Accomplishments That We're Proud Of

From a spark of an idea to a functioning MVP in a single day!!!
The fine-tuning of Microsoft Phi-2, leveraging a trove of medical data, stands as a milestone in our journey, marking a leap forward in AI-powered healthcare.

🔮 Next Steps

🛳️🛳️🛳️🛳️ ship....

Built With

• apple-health
• flask
• intel-developer-cloud
• intel-modin
• prediction-guard