Trial Pulse

Homepage

Inspiration

Clinical trials fail quietly. When a trial can’t enroll enough patients, it closes early: years of work are lost, the patients who did join contributed for nothing, and treatments are delayed. This isn’t rare. In a study of trials on ClinicalTrials.gov, insufficient accrual was the leading reason (57%) among trials terminated for reasons other than their own data (Williams et al., PLOS ONE, 2015). Among 2,579 phase 2–3 trials that closed in 2011, 19% failed to reach their enrollment goal, and those trials had already enrolled 48,027 patients (Carlisle et al., Clinical Trials, 2015).

Most tools attack this from the patient side, helping one person find one trial. We asked the other question: which trials are about to fail for lack of participants, so hospitals can help them before they close?

What it does

TrialPulse ranks all 2,618 interventional trials currently recruiting in Massachusetts by their estimated risk of stopping early because too few people enroll. It’s built for hospital recruitment offices deciding where to send outreach help.

Ranked list with search and filters (risk tier, condition, phase, sponsor, city), sorting, CSV export and shareable links Trial pages showing the risk score as a multiple of the typical trial, the three design features that drove it, eligibility criteria word for word, Massachusetts sites on a map, and a live status check against ClinicalTrials.gov A Massachusetts map shaded by each city’s share of high-risk trials About, Ethics and FAQ pages that show exactly how the model was tested and where it falls short

It ranks trials, never people. It uses no patient data, and eligibility is always decided by the study team.

How we built it

Data: about 380,000 trial records from the ClinicalTrials.gov API v2: terminated and withdrawn trials, completed trials, and trials recruiting today at a Massachusetts site. Labels: we classified each stopped trial’s free-text “why stopped” reason with a keyword rule, then hand-checked 125 labels: 90% of trials the rule flagged were confirmed enrollment failures. Features: only facts known when a trial is designed (phase, sponsor type, number of sites, eligibility rules, age range). Never results or actual enrollment. Model: LightGBM, with a logistic regression baseline and SHAP for per-trial explanations. We trained on trials that started 2008–2018 and tested on 30,087 trials that started 2019–2021, so the test mimics predicting the future. Website: a static HTML/CSS/JavaScript site with Leaflet maps, hosted on GitHub Pages. No backend, so the demo can’t go down. We used Claude Code throughout for building, debugging and auditing, and checked every number against the data ourselves.

Challenges we ran into Data leakage. We found and removed three ways the model was secretly “seeing the answer”: actual enrollment counts, actual end dates, and trials registered after they had already finished. Fixing them lowered our score from 0.79 to 0.76 AUC. We report the honest, lower number. Messy labels. “Why stopped” is free text. Reasons like “slow recruitment and funding” are ambiguous, so we excluded them from training and published our hand-check results. A misleading map. Boston has 2,195 of the 2,618 trials, so raw counts just showed where trials are. We switched to the share of high-risk trials per city, shading only cities with at least 10 trials.

Accomplishments that we’re proud of Results in line with the research literature, honestly measured. AUC 0.76 overall and 0.75 on Massachusetts trials. The 10% of trials TrialPulse flags are 3.5× more likely to fail for low enrollment than average, and checking just that top 10% finds about a third of the trials in trouble. A fully reproducible pipeline. Anyone can clone the repo and rebuild every number. A real finding: the high-risk tier is 95.8% academic-sponsored and 61.8% single-site trials, against 47.6% and 26.2% overall. Boston’s trials are riskier per trial (10.8% high-risk) than Worcester’s (2.6%) or Springfield’s (3.9%). That’s exactly where extra outreach help belongs. Ethics built in, not bolted on. A team member took the project to the hackathon’s ethics review, and we added their feedback (demographics and socio-economic barriers our data can’t see) to our limits and roadmap.

What we learned A simple model (logistic regression, 0.752 AUC) performs almost as well as a complex one (LightGBM, 0.763), and bagging and blends landed in the same place. The limit is the information in the registry, not the algorithm. Removing leakage makes a model’s score look worse but its predictions better. An honest number beats an impressive one. How a score is presented matters as much as the score. A “high risk” flag should bring a trial more help, never be a reason to cut it.

What’s next for TrialPulse Nightly re-scoring, so recruitment offices see new risks the day a trial is registered Stronger signals: each trial’s original enrollment target (recovered from registry version history), competing trials recruiting the same patients nearby, and sponsor track records Community context: the CDC Social Vulnerability Index and MassGIS environmental-justice data around trial sites A pilot with one hospital recruitment office under a written use agreement, to measure whether earlier outreach actually saves trials