Gemini-Powered Holistic Patient Insight Engine

Our platform leverages Gemini to turn unstructured clinical notes into structured mental and social risk data, driving a predictive readmission model and visual dashboard

Comment

Inspiration

Hospital readmissions cost the US healthcare system tens of billions of dollars annually.

Traditional clinical prediction models rely almost entirely on structured EHR data such as:

  • Vitals
  • Lab results
  • Demographic information

However, we realized that the true drivers of hospital readmissions are often hidden inside the unstructured free-text of clinical discharge summaries.

A patient may have stable hemodynamics yet still face:

  • Food insecurity
  • Housing instability
  • Severe anxiety
  • Medication non-adherence

—all of which drastically increase their probability of returning to the Emergency Department.

Our inspiration was to bridge this gap:

Could Large Language Models act not as chatbots, but as deterministic feature extractors capable of quantifying the “unquantifiable” human elements of healthcare?

What it does

Gemini-Powered Holistic Patient Insight Engine

The Gemini-Powered Holistic Patient Insight Engine is an advanced Machine Learning pipeline that demonstrates how unstructured psychosocial data significantly improves hospital readmission prediction accuracy.

🔬 Synthetic Data Engine

The platform generates highly realistic synthetic patient cohorts with correlated:

  • Physical features
  • Psychosocial features

This ensures:

  • Safe experimentation
  • Full HIPAA-compliant testing workflows

🧠 NLP Feature Extraction

The engine uses Google Gemini 2.0 Flash through zero-shot prompting to parse raw physician discharge summaries.

It extracts:

  • SDOH Risk
  • Cognitive Concern
  • Medication Adherence Risk
  • Anxiety Score
  • Additional psychosocial indicators

and converts them into structured integer vectors.

Mathematically:

$$

Z_{\text{extracted}}

f_{\text{gemini}} ( \text{Clinical Notes} ) $$

⚙️ Multi-Model ML Suite

The project trains and cross-validates multiple predictive models including:

  • Logistic Regression
  • Random Forest
  • Gradient Boosting
  • Soft-Voting Ensemble

Two separate pipelines are evaluated:

Suite A — Baseline Model

Uses only traditional vitals and structured EHR features.

Suite B — Enhanced Model

Uses:

$$ X_{\text{structured}} + Z_{\text{extracted}} $$

to generate a holistic prediction model.

📊 Explainable AI Dashboard

The Streamlit dashboard provides:

  • Population health analytics
  • ROC/AUC comparison curves
  • Permutation Feature Importance
  • Calibration plots
  • Clinical intervention recommendations

The interface transforms highly technical ML outputs into intuitive healthcare visualizations.

How we built it

We architected the entire platform in Python, designed specifically for secure local execution.

🧠 LLM Layer

We utilized:

  • google-generativeai
  • Gemini 2.0 Flash

to perform deterministic NLP extraction.

Strict JSON-schema prompting was used to force structured outputs.

import google.generativeai as genai
import json

def extract_features(clinical_text, api_key):

    genai.configure(api_key=api_key)

    model = genai.GenerativeModel(
        'gemini-2.0-flash'
    )

    prompt = f"""
    Extract clinical risk factors.
    Return strict JSON only.
    """

    response = model.generate_content(
        prompt,
        generation_config={
            "response_mime_type":
            "application/json"
        }
    )

    return json.loads(response.text)

⚙️ ML Pipeline

We built the predictive engine using scikit-learn.

Core techniques included:

  • StandardScaler normalization
  • StratifiedKFold cross-validation
  • Permutation Feature Importance
  • Ensemble Voting Classifiers

Our holistic prediction equation becomes:

$$

P_{\text{holistic}}

\sigma ( W_1 X_{\text{structured}} + W_2 Z_{\text{extracted}} + b ) $$

🎨 Frontend

We developed a reactive glassmorphism-style UI using:

  • Streamlit
  • Plotly

Visualizations include:

  • Radar charts
  • Parallel coordinates
  • ROC curves
  • Confusion matrices
  • Calibration plots

🛡️ Fault Tolerance

We engineered a graceful fallback system capable of intercepting:

$$ 429\ \mathrm{Quota\ Exceeded} $$

API failures.

If quota exhaustion occurs, the application automatically pivots to deterministic mock generation so the demo remains fully operational.

Challenges we ran into

🔥 Non-Deterministic LLM Outputs

LLMs are inherently probabilistic.

However, ML classifiers require perfectly structured numeric vectors.

If Gemini outputs malformed JSON or textual hallucinations, the pipeline fails.

We solved this through:

  • Strict JSON schema enforcement
  • Validation layers
  • Integer casting
  • Hard bounded scales:

$$ SDOH \in [1,10] $$

⚡ API Rate Limiting

Processing hundreds of clinical notes sequentially triggered API rate limits.

We solved this through:

  • Controlled request pacing
  • Deterministic fallback systems
  • Local mock pipelines for demos

Accomplishments that we're proud of

🎯 Proving the Hypothesis

We successfully demonstrated measurable AUC Lift when psychosocial NLP features were injected into traditional medical models.

🧠 Ensemble ML Architecture

Rather than relying on a single algorithm, we engineered a fully cross-validated ML suite combining:

  • Random Forests
  • Gradient Boosting
  • Logistic Regression
  • Voting Classifiers

📊 The Dashboard UI/UX

We are incredibly proud of the dashboard experience.

Complex statistical concepts such as:

  • ROC Curves
  • Calibration Plots
  • Confusion Matrices
  • Feature Importance

are transformed into intuitive visual insights usable by hospital administrators and clinicians.

What we learned

Through this project, we learned an important industry insight:

LLMs are exceptional data translators.

The current AI ecosystem heavily focuses on conversational assistants.

However, we discovered that one of the strongest enterprise use-cases for Gemini is:

  • Structured extraction
  • Semantic feature engineering
  • Multimodal preprocessing

Using LLMs as translators between messy real-world text and deterministic mathematical feature spaces unlocks entirely new ML capabilities.

What's next for the Holistic Patient Insight Engine

📚 Few-Shot Prompting + RAG

Our next iteration will integrate:

  • Few-shot prompting
  • Retrieval-Augmented Generation (RAG)

We plan to inject official clinical guidelines such as:

  • PHQ-9
  • GAD-7

directly into Gemini's context window.

🏥 FHIR Integration

We aim to integrate:

$$ HL7\ FHIR $$

standards for direct interoperability with production EHR systems such as:

  • Epic
  • Cerner

⚡ Asynchronous Inference

We plan to convert the extraction pipeline into a fully asynchronous architecture capable of processing:

$$ 1000+ $$

patient records simultaneously.

💻 How to Run the Project Locally

Judges: You can test this pipeline yourself in under 2 minutes.

1. Install Dependencies

Ensure you have Python 3.9+ installed.

Run:

pip install -r requirements.txt

2. Launch the Application

streamlit run app.py

3. Test the Engine

Open your browser:

http://localhost:8501

⚡ Mock Mode

Leave the API key blank.

Set:

$$ 100\ \text{patients} $$

and click:

Run Full Pipeline

The application will demonstrate the full ML workflow in under:

$$ <5\ \mathrm{seconds} $$

🧠 Live Gemini Mode

Paste your Google Gemini API Key into the sidebar.

The engine will perform real-time inference using:

$$ \mathrm{Gemini\ 2.0\ Flash} $$

📂 Custom Dataset Mode

Toggle:

Upload Custom CSV

to run the pipeline on your own healthcare dataset.

Built With

  • clinical-risk-modeling
  • deep-learning
  • ensemble-learning
  • explainable-ai-(xai)
  • glassmorphism-ui
  • google-gemini-api
  • gradient-boosting
  • healthcare-ai
  • hipaa-compliant
  • json-schema-prompting
  • llms
  • logistic-regression
  • machine-learning
  • matplotlib
  • natural-language-processing
  • numpy
  • pandas
  • permutation-feature-importance
  • plotly
  • predictive-analytics
  • python
  • pytorch
  • random-forest
  • scikit-learn
  • standardscaler
  • stratifiedkfold
  • streamlit
  • synthetic-data-generation
  • testing
  • voting-classifier
Share this project:

Updates