Serum Protein Signature for Lung Fibrosis Progression

Inspiration

Pulmonary fibrosis is usually diagnosed late, and it's hard to know which patients will get worse. Many patients first appear at a visit unrelated to fibrosis, which is exactly when an early warning would matter most. We asked whether a single blood draw could provide it, by finding a small set of serum proteins that predicts who is likely to progress.

How we built it

Data: 1,000 patients with first-visit clinical data (age, sex, smoking, BMI, FVC, DLCO), 2,944 serum proteins (Olink Explore 3072), and a follow-up visit about a year later.
Target: progression, defined as a relative FVC drop above 10% or a DLCO drop above 15% at one year.
Pipeline: Python with scikit-learn, organised as Jupyter notebooks that share one utility module.
Leakage-safe preprocessing: all preprocessing runs inside pipelines and is fitted on training data only. This includes custom transformers for missing values and below-detection-limit imputation.
Evaluation: nested, repeated cross-validation (5 × 5). Models are compared on identical folds with a corrected paired t-test.
Signature: the top k proteins along a LARS path, selected inside each fold, tested with 5 to 50 proteins.

Challenges

Data leakage: a group label partly encoded patient outcome, so we removed it at load time and added checks to make sure it never came back.
More proteins than patients: with 2,944 proteins and about 800 training patients per fold, unregularised models overfit badly.
The wrong question at first: predicting next year's FVC gave R² = 0.92, but mostly because next year's value resembles today's. Measured on the change, R² fell to 0.45, and we reframed the project around progression.
Noisy signatures: choosing proteins directly for the outcome overfit. Choosing proteins that reflect disease severity gave a smaller, more stable signature.
Compute: nested cross-validation over thousands of proteins crashed on a shared cluster until we made parallelism configurable.

What we learned

Always check what the baseline already explains before trusting a high score.
Leakage can hide in labels, in preprocessing and in hyperparameter tuning.
A small signature that is stable across folds is more useful than a large one that changes every time.
Proteins carry real signal, but they work best combined with simple clinical data.
External validation in real patients is the essential next step.