Memory Hugs

Memory Hugs shines an early light on Alzheimer’s—and explains the “why”—so families can act sooner, together.

Comment

Memory Hugs 💗

Why this matters

  • 55M+ people are living with dementia worldwide today — projected to reach 139M by 2050. Alzheimer's Disease International
  • In the U.S., an estimated 6.9M adults 65+ are living with Alzheimer’s today, potentially growing to 13.8M by 2060. PubMed
  • 11M+ family caregivers provide about 18.4B hours of unpaid care every year, often with little support. PubMed

Alzheimer’s isn’t just memory loss — it reshapes families, daily routines, and emotional lives.

Memory Hugs 💗 is our attempt to build something practical and demo-ready:
a multimodal Alzheimer’s screening + monitoring dashboard that’s CPU-friendly, interpretable, and designed for “test input → run → outcome”.

What it does

🧠 MRI Classification (4-class)

  • Runs CNN-based MRI classification
  • Compares MobileNetV3 vs ResNet18 fairly on the same validation split
  • Saves per-sample probabilities so everything is auditable and reproducible

🧬 Genetic Variants Modeling (9-class)

  • Trains a fast tabular model on genetic-variant features
  • Includes sanity checks (permutation baseline + cross-validation)
  • Exports per-sample probabilities for inspection and dashboard reporting

🗣️ Speech/Text Risk Scoring (optional pipeline)

  • Supports a lightweight text baseline (TF-IDF + logistic regression style)
  • If a pipeline artifact exists (*_pipe.joblib), Streamlit can do live text inference
  • If not, it still works via saved probability artifacts

⏱️ Early Warning / Care Assistant (monitoring over time)

  • Continuous monitoring view: per-subject dropdown
  • Timeline of risk trend over time
  • Includes a simple uncertainty proxy (entropy-style) alongside risk
  • Can export interactive plots to HTML artifacts that Streamlit embeds directly

🧪 Interpretability + Trust

  • Reliability diagrams (ECE) to measure calibration
  • “Most confident wrong” audits to find the scariest failure cases
  • TTA uncertainty (test-time augmentation) for sample-level trust signals
  • Occlusion sensitivity maps for MRI (model-agnostic XAI)

Demo: “Test input → Run → Output”

We built the Streamlit app so judges can immediately:

  • select a model artifact (MRI / variants)
  • choose a sample index
  • see true label, predicted label, confidence, entropy, margin
  • view calibration plots, XAI images, and interactive HTML charts
  • try text input (if a pipeline is present)

Items

  • Streamlit home/overview page (artifact detection)
  • MRI tab: metrics + calibration + sample inspector
  • XAI tab: occlusion map or Grad-CAM if you saved it
  • Early Warning tab: per-subject timeline and the exported HTML view

How we built it

Data sources

  • MRI dataset: grayscale MRI images stored as bytes (4-class labels)
  • ALZ_Variant dataset: preprocessed genetic/tabular arrays (9-class labels)
  • Speech/Text: NLP baseline (pipeline saved as joblib), timeline simulation for monitoring demos
  • Kaggle tabular Alzheimer dataset

Modeling

  • MRI: MobileNetV3_small and ResNet18
  • Variants: fast tabular model with CV + permutation baseline sanity check
  • Speech/Text: TF-IDF + logistic regression style classifier

Why artifacts matter

Everything writes to /content/artifacts:

  • *.npz probability dumps for evaluation + Streamlit
  • *.csv metrics + report numbers
  • *.png reliability/XAI images (optional)
  • *.html interactive charts (TTA, early warning) for embedding

This makes it robust even if Colab disconnects.

Evaluation (numbers from our run)

✅ MRI architecture comparison (validation)

  • MobileNetV3
    • balanced_acc = 0.8447
    • macro_f1 = 0.8020
    • ECE = 0.0887
  • ResNet18
    • balanced_acc = 0.9386
    • macro_f1 = 0.9191
    • ECE = 0.0309

ResNet18 improved both accuracy and calibration, and reduced extreme-confidence errors.

🔥 “Most confident wrong” audit (MRI)

We automatically print the top-5 mistakes with the highest confidence so we can inspect failure modes directly.
This is one of our favorite “trust” features—because high-confidence wrong predictions are the most dangerous.

🧬 Variants + 🗣️ Speech/Text

  • Our probability-based audit reported no “most confident wrong” cases in the saved files.
  • We still include sanity checks (permutation baseline + CV for tabular) because unusually perfect results can indicate leakage.

What makes this special

✅ Intermediate → Advanced

  • CPU-friendly training and evaluation
  • Strong engineering discipline: artifacts, reproducibility, audits
  • Interpretability + calibration baked in (not an afterthought)
  • Monitoring concept (“Early Warning”) is closer to real-world usage

✅ Built for judges

  • It’s not just a notebook: it’s a polished dashboard
  • Supports interactive testing, exploration, and explanation

Challenges we ran into

  • Colab runtime disconnects wiped state → solved by saving artifacts to Drive backup.
  • Some models had different label spaces (MRI 4-class vs variants 9-class) → we added a fusion tab that explains alignment requirements.
  • Plotting issues (Plotly dependency mismatch) → switched to stable Altair/Matplotlib exports and HTML embedding.

Accomplishments we’re proud of

  • A complete end-to-end pipeline: data → models → evaluation → interpretability → dashboard
  • Trust features that go beyond accuracy: calibration, uncertainty, confident-wrong audit
  • Monitoring concept that feels like a real product direction

What’s next

  • True multimodal fusion (requires aligned subjects + consistent label space)
  • Add clinical tabular dataset prediction (demographics/cognitive tests)
  • Add patient-level aggregation for multiple transcripts per subject
  • Expand Early Warning to incorporate real longitudinal records (not simulated)
  • More robust XAI: counterfactual explanations and decision-curve analysis

Tech stack

  • Python, PyTorch, scikit-learn
  • NumPy / Pandas
  • Altair + Matplotlib for plots
  • Streamlit for the demo UI
  • Colab-friendly artifact workflow (/content/artifacts + Drive backup)

Responsible use note

This project is a research prototype for screening and monitoring experiments.
It is not a medical device and should not be used for clinical diagnosis.
Any high-risk prediction should be interpreted as a cue to seek professional evaluation, not as a conclusion.

💗 Memory Hugs is about turning AI into something caregivers can actually use:
not just a score, but a story—what changed, why it matters, and what to do next.

Built With

Share this project:

Updates

Submission history