TrialPilot

TrialPilot helps teams compare clinical trial designs across cost, site workload, patient travel, and timeline risk with transparent, traceable estimates.

Comment

Inspiration

A clinical trial design is a chain of connected decisions. Moving a visit, changing monitoring frequency, or choosing a different supply strategy can affect cost, site workload, patient travel, and completion time.

Regeneron's Choose Your Timeline starter sheet describes the problem: cost, enrollment, and timeline models live in separate spreadsheets, so "each 'what if' takes days." We wanted to make those connections easier to explore, and we used trials of Regeneron medicines as our worked examples. Our starting question was: Can a trial team compare design choices in one place and understand the evidence behind each estimate?

That became TrialPilot.

Pitch deck: https://drive.google.com/file/d/1mVeFFZbcdsejZ8vlddyUB-9J9RnbuxqE/view?usp=sharing

What it does

TrialPilot is an open-source clinical trial planning prototype connecting protocol review, scenario simulation, operations monitoring, and software-validation evidence.

Demonstrated on Regeneron trials

The built-in trial library uses public ClinicalTrials.gov records of trials of Regeneron medicines:

  • BOREAS (NCT03930732) and NOTUS (NCT04456673): the Phase 3 trials of dupilumab in COPD with type 2 inflammation. Dupilumab is developed by Regeneron and Sanofi; the registry lists Sanofi as lead sponsor and Regeneron as collaborator.
  • PHOTON (NCT04429503): Regeneron's Phase 2/3 trial of high-dose aflibercept in diabetic eye disease.

The library also includes one non-Regeneron COPD trial (ensifentrine, NCT04535986), and any other public trial can be imported live by its NCT ID.

Walkthrough: BOREAS

The demo video and pitch deck follow BOREAS from protocol to approval:

  • Protocol review. TrialPilot imports the BOREAS registry record (939 participants, 275 site entries, 24 countries) and flags inputs the registry does not provide. It turns the protocol into 19 visit candidates, each with a page reference and quoted excerpt, which a person reviews before they are applied to the design.
  • Peer context. PeerMirror compares BOREAS with 439 matched Phase 3 COPD registry peers: 17 registered outcomes against a peer median of 6, 24 countries against 2, and 275 site entries against 34. These descriptive counts frame review questions; they do not measure trial quality.
  • Scenario comparison. The simulator explores monitoring, visit-setting, laboratory, and drug-supply choices. For BOREAS with 50% remote monitoring and 50% local labs (upfront supply), the model's P50 cost moves from $330.0M to $322.6M, site staff time from 235,075 to 223,150 hours, and carbon from 1,351 to 1,097 t CO2e, while patient travel and duration do not change. The selected scenario's cost P10–P90 is $190.0M–$597.8M. The paired cost change for adding 50% local labs is −$7.14M / −$2.99M / +$0.35M (P10 / P50 / P90), so the model does not rule out a small cost increase. These are model outputs under stated assumptions, not observed savings.
  • Explaining the numbers. Every estimate opens to its formulas, parameter sources, sensitivity drivers, and assumptions. A read-only MCP server lets an AI assistant answer questions such as "Why is the BOREAS cost range so wide?": the biologic kit price, a market-price proxy, drives about 40% of the cost variance. Saving and signing stay in the web workflow.

Operations, approval, and audit

The command center uses clearly labelled synthetic feeds for a simulated 40-site trial to demonstrate alerts, interventions, and follow-up analysis.

Versioned scenarios, separate reviewer and QA demonstration roles, record-bound signatures, and an audit chain make the decision history inspectable.

How we built it

We built the interface with React and TypeScript, the backend with FastAPI and SQLite, and the simulation engine with Python and NumPy. Shared Pydantic contracts connect the components.

For activity-based components such as cost, workload, and carbon, the underlying calculation is:

$$Y_m^{(k)}(s)=\sum_i A_i(s)\,\theta_{i,m}^{(k)}$$

Here, the scenario determines activity quantities, while sampled parameters represent uncertainty in costs, time requirements, and emission factors. Separate enrollment and supply simulations produce timeline and drug-waste estimates.

Seeded Monte Carlo sampling produces P10, P50, and P90 ranges. Scenario comparisons use paired draws:

$$\Delta_m^{(k)}=Y_m^{(k)}(s_{\mathrm{candidate}})-Y_m^{(k)}(s_{\mathrm{current}})$$

Each comparison uses the same underlying parameter draws, helping isolate the modeled effect of a change. The reported difference ranges come from these paired differences.

Optional protocol extraction uses the Anthropic API. During development, we coordinated AI coding agents through shared interfaces, module responsibilities, and acceptance checks.

Challenges we ran into

The biggest challenge was handling incomplete information honestly. Registry records often lack detailed visit schedules, and extracted protocol text needs human review. We made missing values, assumptions, and review candidates visible throughout the workflow.

We also had to preserve design versions across calculation, saving, comparison, and approval. Otherwise, a user could approve a result calculated from a different design than the one displayed.

Another challenge was balancing simulation detail with memory and latency. Forecast and supply components report their actual sampling budgets so users can inspect how each result was produced.

Finally, integrating the modules required consistent contracts and meaningful regression checks. A working interface needed to remain consistent with the underlying calculations, stored records, and audit history.

Accomplishments that we're proud of

We built a connected workflow from protocol review to scenario comparison, approval demonstrations, and traceable software evidence, and ran it end to end on a real Regeneron trial record.

A documented clean-clone run at commit 8fca2d7 recorded 819 passing checks, 32/32 traced requirements passed, and 115 package files downloaded and hash-verified (the deck appendix shows the earlier 576-check run at e5c386b). Human QA approval remains pending; automated demonstration signatures do not constitute human approval.

In a separate synthetic evaluation across three seeds, the command center detected 42/42 planted operating problems, and 11/12 intervention verdicts matched the simulator's withheld ground truth.

We are especially proud that users can inspect the assumptions behind an estimate. The interface exposes formulas, parameter sources, sensitivity drivers, missing inputs, and model limitations alongside the results.

What we learned

A reproducible calculation can still depend on uncertain inputs or incomplete model boundaries. Against two published trial carbon footprints, CASPS and PRIMETIME, our P50 estimates were 23.4% and 22.8% higher (excluding drug supply), with both published totals inside our model ranges. Those gaps showed us where our activity boundaries differ from the studies.

We learned to distinguish public source data, model assumptions, software test results, and human approval. Each answers a different question.

We also learned that uncertainty needs to be part of the interface. Our ranges describe outcomes under the model's assumptions; real-world accuracy and interval coverage still require evaluation.

Building TrialPilot reinforced the value of shared contracts, versioned records, and checks that follow a decision through the complete workflow.

What's next for TrialPilot

Our next priorities are scenario-specific assumption overrides and evaluation with permitted real study data.

The first override we would add is a sponsor's own investigational-product cost in place of the biologic kit's market-price proxy, the largest driver of BOREAS cost variance. More broadly, we want trial teams to replace library assumptions with their own documented inputs, then measure forecast calibration, extraction quality, and human review effort.

Further development includes authorized operational connectors, persistent hosting, production authentication, and independent QA.

TrialPilot currently uses public registry records and synthetic operating feeds. Our goal is to develop it into a practical planning tool with clear evidence, explicit uncertainty, and qualified human review.

Share this project:

Updates

Submission history