Inspiration

AI that screens retinal photos for diabetic retinopathy is validated once, on one population and one kind of camera, and then deployed to many clinics. When it quietly starts missing disease at one of them (a worn camera, a different patient group, a vendor software update), nobody finds out until expert grades come back: 5 to 7 weeks later, and for only about 1 scan in 10. By then, people with sight-threatening disease have been sent home.

We wanted every clinic to get an early warning, and more than a warning: a clear, defensible decision about whether to pull the AI for review.

The proof arrives 6 weeks late. ClinicDrift doesn't wait.

What it does

Every week, for every clinic, ClinicDrift does two things:

Predict. It estimates the AI's catch rate (sensitivity for referable disease) from the images alone, before any expert grade arrives, with an honest uncertainty band.
Decide. It raises an alarm only when the expected harm of carrying on outweighs the cost of a review. A drift alarm fires when the scans stop looking like the ones the AI was validated on; a harm alarm fires when the scans look normal but the evidence says the catch rate has fallen below the UK screening standard of 80%.

When an alarm fires, Gemini suggests the likely cause and drafts an incident report. Everything is shown in a 3D web app: a globe of clinics, each clinic week by week, live cost sliders that re-decide every alarm, and the backtest.

How we built it

The model. We fine-tuned a ConvNeXt-tiny on 24,718 EyePACS retinal scans (split by patient), calibrated it with temperature scaling and set the referral cut-off on validation to catch 90% of referable cases. Held-out test AUC: 0.909.

The clinics. We replayed 9,500 real scans from four public datasets (EyePACS, Messidor-2, APTOS, IDRiD) as 10 simulated clinics over 20 weeks, including a camera that slowly wears out and a faulty software update that raises the referral cut-off. 10% of scans get an expert grade, arriving 35 to 49 days later.

Label-free estimate. With calibrated probabilities $p_i$ and referral cut-off $\tau$, the expected catch rate in a week is

$$\hat{s} = \frac{\sum_i p_i \,\mathbb{1}[p_i \ge \tau]}{\sum_i p_i}.$$

Drift. A classifier tries to tell this week's scan features from the validated ones; its cross-validated AUC is the drift score (0.5 = indistinguishable). Trust in $\hat{s}$ fades as drift rises:

$$t = \operatorname{clip}!\left(\frac{0.9 - \mathrm{AUC}}{0.9 - 0.7},\, 0,\, 1\right).$$

Belief. The estimate is blended with "anything is possible" in proportion to $1 - t$, turned into a Beta distribution, and updated with every expert grade that has arrived: $\alpha \mathrel{+}= \text{caught}$, $\beta \mathrel{+}= \text{missed}$.

Decision. Over a horizon of $h = 6$ weeks, with $n$ expected referable cases a week,

$$H = h\Big(c_{\text{miss}}\, n\,(s_{\text{ref}} - \bar{s}) + c_{\text{fp}}\,\Delta_{\text{referrals}}\Big),$$

and the AI is pulled when $H > c_{\text{review}}$ and either the scans have drifted ($t < 0.5$) or $P(s < 0.8) \ge 0.8$. Costs are illustrative (£10k per missed case, £150 per unnecessary referral, £25k per review), so we swept them.

Stack. Python (pandas, NumPy, SciPy, scikit-learn, PyTorch) for the engine; Tiger Data (TimescaleDB) for scan events and weekly results; Gemini for cause-finding and reports; React, TypeScript and Three.js for the web app; Streamlit for the dashboard; and deploy scripts for Vultr (planned hosting).

Results

We backtested four ways of monitoring the same clinics, and replayed it 100 times with fresh draws of which scans got graded:

Policy	Changed clinics caught	False alarms	Extra missed urgent cases	Total cost
ClinicDrift	8 of 8, every time	0%	1.0	£210k
Wait for expert grades (today)	0 of 8	18%	29.6	£489k
Watch the AI's own estimate	2 of 8	0%	6.7	£302k
Watch drift only	7 of 8	0%	19.8	£385k

At the worn-camera clinic, ClinicDrift raised the alarm in week 8, one week after the wear began. Watching the AI's own estimate would only have done so in week 16, and waiting for grades never did within the 20 weeks.

Challenges we ran into

No single signal catches everything. Drift can't see a software update (the scans look identical), and the AI's own estimate is too noisy to act on. Only combining them, with the few grades that arrive, caught all eight changes.
Grades are late and scarce. A clinic of 50 scans a week has only a handful of graded urgent cases to go on, so the belief has to be honest about uncertainty, and alarms latch until someone reviews them, or old grades would switch them off just as harm peaks.
Tuning on real results, openly. Our harm alarm first required 90% certainty, a bar set on an early stand-in model. On the trained model it held the software-update alarm back to week 19, so we lowered it to 80%: the alarm came in week 12, still with no false alarms in any replay. We say so in our results, and treat the backtest as tuned on this replay rather than a fresh test.
Saying only what we know. On a drift alarm the catch rate genuinely can't be verified, so the alarm and the reports say "can't verify", never a made-up number.

What we learned

Outside its validated conditions, a model's performance can't be read off its own scores. In France and India our model turned out to catch nearly every urgent case, but sent 71% of healthy people to an eye clinic. Nobody could have known which way it would go.
Putting a price on missed cases and on reviews turns "is the AI still OK?" into a decision a clinic lead can act on and argue with.
Honest limits make a stronger case: the clinics, camera wear and update are simulated; the costs are illustrative; and the datasets carry no age, sex or ethnicity, so we make no fairness claims about patient groups.

Deployment (Vultr)

Vultr was unavailable at the hackathon, so this is our deployment plan, with the scripts already in the repo.

3D web app: built and served on port 8080 by deploy/vultr-web.sh.
Dashboard: on port 80 via deploy/vultr-dashboard.sh, reading live from Tiger Cloud.
One-paste deploy: both run from Vultr's Cloud-Init straight from GitHub, as self-restarting services.
Next: one VM per health board in Vultr's London region, with the engine's weekly run scheduled on the same machine.