GlucoGuard

Every model compared at the same false-alarm rate. Linear extrapolation looked best at a fixed 70 mg/dL cutoff
Accuracy improves left, low-catching improves up, and the models go the wrong way. Selecting on RMSE ships the worst alarm.
Step one, and the disclosure up front: this wearer is simulated, the model is the real trained checkpoint
The app read the trace, decided this was worth an interruption, and sent the notification itself. Nobody pressed anything.
In range, and saying so quietly. Predicted 132 mg/dL in 30 minutes, 0% chance of going low, nothing to do.
The idea in one screen. You set how often it may be wrong; GlucoGuard fits the cutoff that delivers it from your own first two weeks.
A full day replayed
All eleven models on the same eight held-out wearers. Read it sideways: the RMSE column and the low-glucose column disagree almost perfectly

Inspiration

A continuous glucose monitor is a very good sensor and a very late alarm. It measures where blood sugar is right now, so when it warns about hypoglycemia you are already hypoglycemic. At 3 a.m. that alarm is also competing with sleep, and severe nocturnal lows are among the outcomes people with type 1 diabetes fear most.

The gap is not sensing. It is anticipation. We wanted to know how much warning you can extract from the CGM signal alone: no new hardware, no extra inputs, just the trace the device is already producing.

What it does

GlucoGuard reads the last two hours of CGM and predicts a distribution over blood glucose 30 minutes ahead. That distribution drives one decision: what is the probability this person goes below 70 mg/dL?

On eight held-out wearers it warns about 77% of low-glucose episodes before they begin, at six false alarms a day, with a median of 25 minutes of warning.

The alarm threshold is not a constant. It is fitted to each wearer from their own first two weeks, because a single population-wide cutoff delivers wildly different alarm rates to different people: 3 to 26 a day across our external cohort.

It does not recommend insulin doses.

How we built it

Data. The OpenAPS Data Commons: a 6.5 GB archive of Nightscout exports donated by people running open-source automated insulin delivery. We read CGM records straight out of the zip, kept sensor glucose only, clipped to a plausible 20-450 mg/dL, and resampled onto a regular 5-minute grid. That gives 40 wearers, 8.1 million readings, 28,281 patient-days.

The split is by patient, never by row. This is the decision the whole project rests on. CGM samples five minutes apart are enormously autocorrelated; split rows at random and the model memorizes a trace it saw a few steps earlier, producing a beautiful test score and a system that fails on the first new person it meets. 26 wearers train, 6 select, 8 are held out entirely.

Models. Persistence and linear extrapolation as non-learned floors, ridge regression on the raw window, then LSTM, dilated TCN, and Transformer, all predicting the change in glucose rather than the level, with rate of change as an explicit second channel. Then a probabilistic head (Gaussian NLL), a classification head trained directly on the low/not-low label, and loss re-weighting toward lows.

The alarm is a separate layer. Each model emits a risk score; the cutoff on that score is tuned on validation to a false-alarm budget, then applied unchanged to test. Alarms fire once and stay quiet for 30 minutes, and are scored on episodes rather than readings, because that is what a wearer experiences.

Challenges we ran into

We caught our own evaluation lying to us, four times. Each one changed the project more than any modeling choice did.

Ranking by RMSE almost exactly reverses the ranking by low-glucose recall. Our most accurate model was the worst at catching lows, worse than doing nothing. Squared error rewards a forecast that hugs the mean and hypoglycemia is the tail, so optimizing accuracy taught the model to refuse to commit to the exact events we built it for. Selecting on RMSE, as is standard in this area, would have shipped the worst available alarm.

Reading every model at a fixed 70 mg/dL cutoff compares their biases, not their skill. Linear extrapolation reached 74% recall by alarming 21 times a day. Matched to the same false-alarm budget it comes last.

Our first lead-time metric counted warnings issued after glucose had already crossed 70. The median came out as exactly zero, which is what exposed it.

We were choosing the shipped model by reading test recall. The top three sat within 1.6 points of each other, exactly where selecting on test turns noise into a decision. Selection moved inside validation, using two patient folds.

And the archive fights you. Two incompatible export formats, one wearer holding 1.5 GB alone, traces riddled with dropouts. We stream the relevant files out of the zip, interpolate gaps up to 15 minutes, and discard any window spanning a longer one: filling a three-hour hole with a straight line invents data and quietly inflates the score.

Accomplishments that we're proud of

The evaluation is the accomplishment, and its best moment was catching a contamination nobody would have found by accident. We built a second cohort from the untouched half of the archive: 33 AndroidAPS wearers, different app, different sensors, mostly European. While assembling it we discovered that four donors had uploaded under both export formats, one of them a test wearer. Without that check, our "external validation" would have been a re-test on people the model already knew.

Applied unchanged to that population, the model holds: RMSE 19.95 against persistence's 23.77, Clarke A+B 97.3%, and at every matched false-alarm rate it still beats persistence by 9 to 13 points. The ranking survives; the calibration does not, which is precisely the evidence that sent us to per-wearer thresholds.

What we learned

That the hard part of medical machine learning is the measurement, not the model. Every meaningful decision came from finding a place where our own evaluation was being too kind to us.

The clearest example arrived last. Adding insulin and carbohydrate records improved validation RMSE, made test RMSE worse, and improved the alarm on both. We concluded "the extra inputs failed" from RMSE, and had to withdraw that an hour later after scoring them as alarms. It is the exact mistake this project exists to warn about, committed by us, on our own work.

What's next for GlucoGuard

Separate SMB from meal boluses. One wearer logs 71 boluses a day at a median of 0.20 U; another logs 0.5 a day at 3.5 U. The first is a loop micro-dosing, the second is a person eating. The same number in the same channel means opposite things, and that is the likeliest reason treatment inputs transfer badly between cohorts.
Withhold predictions on out-of-distribution input. The predictive spread already exists; it should gate the output, not just decorate it.
A prospective evaluation. Everything here is retrospective replay.
Longer horizons, reported separately rather than quoting the easy one.

Note on prior work: background research (reading the CGM forecasting literature, obtaining the OpenAPS Data Commons dataset) was done before the hackathon, which the rules explicitly encourage. All code in this submission was written during the hackathon period and the repository's commit history reflects that. No pre-existing codebase was submitted.

The hosted demo plays back simulated wearers. The model is the real trained checkpoint and every measured result quoted above is from the real cohort; the trace is generated because the donated recordings are not ours to republish, and the app says so on screen.