CRAP

9 in 10 clinical trials fail due to its tedious process resulting in a depleted budget and fatigue. CRAP automates this. It identifies weakness in FDA compliance and trial before it is an issue.

Comment

Inspiration

CRAP started with a very personal story: one of our teammates watched his sister battle cancer, and it was a battle she barely survived (We have her family's permission to share this). Part of what made that fight unnecessarily hard was the fact that the science that could've made the battle super easy was stopped due to paperwork issues. She survived through a miracle, but most families don't get that news. The science had no issues in terms of saving her sister; it was a slight mistake in paperwork and the protocol. Promising treatments aren't moving at the speed of science; they are forced to work at the speed of paperwork. Clinical trial protocols get delayed, amended, and a lot; 90% of them just get dropped due to expensive legal review cycles, regulatory things, back and forth between the government, etc. This is only getting exacerbated in the future with AI coming in to help research science. We built CRAP to directly speed up that bottleneck as much as possible. If protocol review is sped up, science is able to move at a much faster pace, and that's our mission.

What it does

CRAP (Comprehensive Review and Analysis Platform) analyzes and improves clinical trial protocols. You can submit a Phase II–III protocol PDF, and it will structure it into machine-readable CDISC USDM v3.0(endpoints, arms, schedule of activities, and eligibility). Then it will benchmark your protocol against 591,000. It finds the most similar trials and gets duration, enrollment, and burden baselines. It scores participant recruitment difficulty, how many people are eligible, it reviews eligibility rules, predicts duration overrun risk with a custom trained XGBoost model, explained per prediction with SHAP, checks FDA compliance against a library of 12 real FDA guidance documents, flags gaps, and then it regenerates an improved protocol draft with citation-linked recommendations, exportable as USDM JSON/XML, for CRO/CTMS integration. It effectively automates the process of reviewing the whole protocol manually, from reviewing what the agent flags and accepting its recommended edits, and it automates the process of finding all the potential issues, the process of finding similar trials, etc.

It also includes a natural language search over the full 591k trial database, wrapped in a dashboard that streams pipeline progress live via WebSocket. That same trial database is also exposed through our own MCP server, so any MCP-compatible client can query it directly.

How we built it

The backend is FastAPI, running an 8-stage asynchronous pipeline, streaming progress to a Next.js frontend over multiple WebSockets. First, it parses the PDF, using a hybrid of PyMuPDF and pdfplumber, from the protocol document that was uploaded. Then we use DeepSeek to transform the text into a structured USDM JSON that we can use more easily; lastly, we use sentence transformer embeddings and 4-factor similarity scoring (condition 35%, endpoints 25%, phase 20%, design 20%) against 591K trials in SQLite. Fourth, it'll use a set of weighted benchmark metrics aggregated from the top matches, and then it'll assess rule-based participant burden and recruitment scoring. Then our custom XGBoost model trained on 556k completed trials will predict where risks may arise, and then our 2-stage FDA analysis happens, where Deepseek selects the relevant guidance documents, and performs the gap analysis. Lastly, it generates an improved draft with citation-linked recommended changes, each one tied back to a similar trial's NCT ID or an FDA guidance document, so every suggestion is reviewable and exportable as USDM JSON/XML.

Challenges we ran into

The ML pipeline was very difficult, and so were our matches. Training on 556k trials, generating sentence embeddings for the full dataset, kept killing our personal laptops. Our memory was pushed to its limits; they were overheating, and we crashed multiple sessions (5 times we crashed), and lost a lot of time and progress to the ML pipeline; however, we didn't give up on it. We moved on to a better computer, and we built embedding caching into the pipeline to make it so that crashed runs can resume instead of starting over from square 1. Beyond compute, data sanitization also gave us a hard time. 591,000+ trials are no joke, and a lot of them contain edge cases you may not have accounted for that can slowly ruin our model, but we caught them and sanitized all of the data with scripts.

Accomplishments that we're proud of

We are proud of 2 main things: we trained our own custom XGBoost duration-risk model on over half a million real datapoints, with per-prediction SHAP explanations so every risk score is readable; it isn't a black box. We are also very proud of our real workflow integration. CRAP isn't a demo that just ends as a dashboard; it exports CDISC USDM v3.0 JSON/XML, real, industry-standard formats, so industry professionals can use it directly in their workflows; they can plug it into existing CRO and CTMS workflows. Our product is something clinical operations teams could genuinely use in their workflows to automate a lot of work.

What we learned

We learned a lot, especially about machine learning and data sanitization; training on hundreds of thousands of real records taught us a lesson about memory management, knowing your hardware, checkpointing, cache, and when to stop, and try a new approach (we stopped optimizing our ML pipeline and moved to a real GPU). We also learned how real medical registry data is messy, and we hadn't dealt with that as much in curated datasets such as those available on Kaggle. We were also taught a lot about how the medical industry handles protocols; we dived into FDA guidance documents and USDM standards, and we were shown just how much of clinical research is, in reality, just a lot of paper pushing and legal boilerplate, which, even with slight mistakes, costs millions to sponsors, and for some it costs their lives. Protocols are stalled due to legal issues, which makes no sense, and it showed us the inefficiencies in science due to politics and paperwork.

What's next for CRAP

3-month goals:

Expand our MCP server so any MCP-compatible AI client can query the trial database and run protocol analysis directly Grow the FDA guidance library beyond the initial 12 documents Pitch CRAP to real people, such as CROs, academic research coordinators, and clinical operations teams, for feedback on our design Improve our ML model and increase past the 0.3 R2 coefficient

6-month goals:

Land our first partnerships with research institutions and incorporate real reviewer feedback into the platform Improve the duration-risk model with richer features and site-level data Support Phase I and Phase IV protocols 1-year goals:

Become part of the standard pre-submission workflow for a portion of the industry Full CTMS integrations with major platforms Expand beyond FDA to EMA and other international regulatory frameworks, and other countries as well

Built With

Share this project:

Updates