-
-
01-cover.png: Does a breast cancer prognosis hold up in another hospital? Ranking holds at six sites; the level of risk does not.
-
02-predict-evidence-verdict.png: Predict page: a risk estimate together with the evidence verdict behind it.
-
03-predict-estimate-withheld-dark.png:An unusual profile:the estimate is withheld because too little evidence supports it (dark theme).
-
04-ranking-transports.png: Held out Uno C at each of the six sites, pooled with a prediction interval.
-
05-level-of-risk-does-not.png: Observed over expected deaths at each site: the level of risk does not transport.
-
06-censored-is-not-alive.png: Label audit: counting censored patients as survivors understates mortality.
-
07-hospital-local-check.png: Hospital page on the built in example file: does the level of risk hold up locally?
-
08-audit-rederived-in-browser.png: Methods page: the audit of 24,624 input profiles, re-derived in the viewer's browser.
-
09-works-on-a-phone.png: Predict, Evidence and Label audit at phone width.
Inspiration
Every prognosis model looks great on the data it was born from. Almost none show what happens in the next hospital.
We wanted to ask the question nobody wants to ask: does it actually hold up? So we took TCGA-BRCA from the NCI Genomic Data Commons plus five METABRIC cohorts, six sites in all, and locked each one out in turn.
Then there was the quiet problem. In TCGA-BRCA the median follow-up is 30 months and 56% of patients have no known 3-year status. Call those patients "alive" and the numbers look better than the truth. We refused to.
What it does
GDC Lens tells you what a breast cancer prognosis is worth before you trust it. No server. No sign-up. Everything runs in your browser.
- Predict. Risk of death by 3, 5 and 10 years from age, pT, pN, ER, PR and HER2. Every estimate carries a verdict on its own evidence: show, show with caution, or withhold it. It is willing to say "I don't know."
- Evidence. Held-out results at every site, pooled with a prediction interval, with calibration and decision curves.
- Label audit. Exactly how much "censored = alive" lies to you, with a simulation you can run in the page.
- Hospital. Drop in a CSV of your own patients. Get the same checks, a local correction and a report. The file never leaves your browser.
- Live GDC. Re-runs the headline checks on current records from the GDC API with a model that never saw TCGA.
- Methods. The full model, the measures, the limits, and an audit of all 24,624 input profiles that you can re-derive in your own tab.
How we built it
A Cox model written from scratch in NumPy, so the exact same arithmetic runs in Python and in the browser. Risk at horizon (h):
$$ \hat P(T \le h \mid x) = 1 - \exp{-\kappa\, H_0(h)\, e^{\beta^\top x}} $$
with (\kappa = 1) for the pooled model. The Hospital page estimates a local factor (\hat\kappa = O/E), observed over expected deaths, with an exact interval.
Censoring is handled properly: Kaplan-Meier (\hat S(t)=\prod_{t_i\le t}\bigl(1-d_i/n_i\bigr)), reverse Kaplan-Meier (\hat G(t)=\hat P(C>t)) for follow-up, and inverse-probability-of-censoring weights. Ranking is scored with Uno's C
$$ \hat C_\tau=\frac{\sum_{i,j}\Delta_i\,\hat G(T_i)^{-2}\,\mathbb 1(T_i\eta_j)}{\sum_{i,j}\Delta_i\,\hat G(T_i)^{-2}\,\mathbb 1(T_i<T_j,\;T_i<\tau)} $$
and level of risk with the censoring-weighted Brier score, observed over expected deaths, calibration slope and ICI. Sites are pooled with DerSimonian-Laird and Hartung-Knapp, plus a 95% prediction interval for the next site: (\hat\mu \pm t\sqrt{\hat\tau^{2}+\mathrm{SE}(\hat\mu)^{2}}).
The site is plain JavaScript, Vite and hand-built SVG charts. No framework, no backend. Hypotheses and decision rules were written down before any cross-site model was fitted, and every change since is logged.
Challenges we ran into
- Short follow-up. TCGA-BRCA barely follows patients past 30 months. Every measure had to be censoring-aware from day one.
- Six sources, one definition. Staging, receptors and follow-up differ across sources. 3,607 patients became 2,811 after documented exclusions. Sex is empty for every TCGA-BRCA case in GDC release 46, so we dropped it rather than fake it.
- Two codebases that must agree to the decimal. Browser and Python are tested against each other on Kaplan-Meier, weights, Uno C, AUC, calibration, the local factor, the model on 150 profiles, the audit on all 24,624 profiles, and the Hospital page.
- Publishing the results that didn't work. The evidence check did not locate calibration failures. The local correction hurt at 4 of 6 sites. They are in the results exactly as they came out.
- Phones. Forest plots and tables got their own narrow layout.
Accomplishments that we're proud of
- A result that survives scrutiny: ranking carries over (pooled Uno C 0.69, 95% interval 0.65 to 0.73), the level of risk does not (observed over expected deaths from 0.51 to 1.60 across sites).
- Caught the censoring lie in numbers: 3.8% mortality counting censored as alive, 8.6% dropping them, 6.1% by Kaplan-Meier.
- A small Cox model that holds its own against a random survival forest and gradient-boosted survival on ranking.
- Verified, not vibes: 29 Python, 24 JavaScript and 14 browser tests in CI, Kaplan-Meier and Cox checked against statsmodels and lifelines, and automated WCAG 2.2 AA checks in light and dark.
- Every table reproducible from the open repository with the commands in the README.
What we learned
- Ranking transports. Level of risk does not. A model can sort patients well and still be badly off about how many will die.
- Censored is not alive. The label you pick can move a 3-year mortality estimate by several points.
- An evidence check describes the data, not the errors. Per-profile verdicts did not locate where calibration failed (difference 2.0 points, permutation (p = 0.40)).
- A one-number local fix is not free. Rescaling from 100 local patients helped at 2 sites and hurt at 4.
- A reassurance threshold chosen on other sites held: at or below 5% observed 3-year mortality at 5 of 6 held-out sites.
- Negative results make the positive ones believable.
What's next for GDC Lens
- Clinical and statistical review. A public review form is open on the repository, and we want hard feedback.
- More cohorts to stress the ranking-versus-level finding further.
- Treatment as a predictor, since it is not modelled yet.
- Death from breast cancer as the endpoint instead of death from any cause.
Research use only. Not a medical device. Not for decisions about individual patients. Not validated prospectively.