Predicting Functional Outcomes of Mechanical Thrombectomy

You can find our comprehensive final write-up at the link below!

https://docs.google.com/document/d/1DhlyKjfbQcb-fGescl42WXB9gXYoU781M96-g71IbuI/edit?usp=sharing

Introduction:

For our DL final project, we knew that we wanted to work on something that would have a tangible impact. Some of our group members had previous experience and interest in medical applications for computer science -- as such, we were drawn to the medically applicable external pitches. We decided to work on this project because we knew we wanted to work with images, and because we were really attracted to the direct applications of the research. Our model, once completed, will be used to predict the outcome of mechanical thrombectomy on a patient with a stroke before the operation even occurs. The project that we are tackling is a supervised image classification problem—we are tasked with taking in CTA and DSA images (essentially X-rays of the skull) and other relevant features about the patient (such as sex, age, and type of injury)—and from these, predicting the TICI score and the number of passes needed to complete a successful operation. The TICI (Thrombolysis in Cerebral Infarction) score is a measure of the amount of blood flow through a blood vessel after the operation has been completed. TICI scores fall into discrete classifications (0, 1, 2a, 2b, 3), which will be spit out by our model.

Related Work:

Fully-Automated Identification and Characterization of Large Vessel Occlusions on CT Angiography with 3D Convolutional Neural Networks I Pan, MA , Providence, RI; T Tran, BS; T Yi; P Wang; S Meng; J Wu; et al.

From the Pan and colleagues paper, researchers established that a 3D CNN was able to identify and classify large vessel occlusions from CT image data into either acute LVO, chronic LVO, or no occlusion classifications at a high degree of effectiveness (>40% of acute LVOs would be identified with a 5% false positive rate). It did so by training and testing the data in an 80/10/10 train/validation/test partition. The paper substantiates that a neural network has the potential to help make the LVO identification process more efficient.

Link to paper

Data:

We will be using a dataset provided to us by Rhode Island Hospital. This dataset consists of 390 actual patient CTA and DSA scans as well as other possibly relevant features like their sex and age. Although it is interesting to be working with live patient data, our group must be careful to comply with HIPAA rules and not mishandle any of the data. The data that we are given is already anonymized, which means that the original subjects cannot be traced back. Preprocessing is partially done; images have already been converted into .npy files. We will need to separate the datasets into appropriate train and test sets. We will also need to extract patient data from a CSV file and relate this to the images. Another challenge with this dataset, because of its nature, is that we are not allowed to work with it on our own personal machines - we must use a remote desktop software to access the hospital’s computer and run all of our code on said machine. As such, we may run into roadblocks when it comes to setting up the environment and making sure that our code runs well on a remote machine.

Methodology:

We envision using a convolutional neural network in order to classify the images. The model will be trained using a 90/10 training/testing split from the data we receive from RIH. The model will be trained analogously to how we train models in homeworks; we will run our model for several epochs on the training data and then on the test data once trained. Because we are working with a discrete classification problem on images, it made sense to us to use a CNN. However, one problem that arises is that the images given to us do not take the same format as the ones we worked with in the assignment. Each scan consists of 64 images, each of which is an image taken from a different perspective of the skull. All the images come together to essentially make a 3D image of the skull. These perspectives replace the RGB channels we are familiar with dealing with in CNNs. As such, we do not know whether we can simply apply standard convolution to each image independently as every “channel” does not preserve the position of the skull. We will need to research if it is possible to somehow apply convolution to 3D images.

Metrics:

Success could be measured in a variety of ways. From our external project coordinator’s point of view, this project is mainly a proof of concept, and our model would be considered successful in their eyes if it performs better than random guessing (i.e. 50% accuracy). Since the application of our development would be to assist physicians as a secondary tool to inform medical decisions, a successful neural network would make predictions about classifications with a relatively high degree of accuracy as well as in a reasonable amount of time. Ideally with a higher degree of accuracy and a lower runtime than what it takes a physician to manually assess the images but beating a physician in either accuracy or runtime (without compromising significantly on accuracy) could be seen as a success.

Some experiments to run could include testing the accuracy rate, average runtime, false positive rate, and false negative rate of our model.

Base goal: >50% accuracy

Target goal: >60% accuracy

Stretch goal: >75% accuracy

Ethics:

What is your dataset? Are there any concerns about how it was collected, or labeled? Is it representative? What kind of underlying historical or societal biases might it contain?

One ethical concern for our project is obviously the dataset. Although we are given anonymized data, at the end of the day, we are still working with data from actual patients who suffered from acute stroke and needed to undergo mechanical thrombectomy. I am confident that whoever runs the RIH research lab obtained this data through the proper channels; however, when I think about whether I would want my doctor’s checkup results to be used in someone else’s machine learning algorithm, I would feel uncomfortable from a privacy perspective. The theme of patient consent and autonomy when it comes to their data being used in DL models is important to consider; it’s one thing for the doctors to be able to view your patient data, but additional consent would be required for use in a program. One possible bias that presents itself in the dataset is a socioeconomic bias -- because this dataset is taken out of RIH patients, perhaps it is only representative of those who could afford treatment at such a good hospital. Additionally, the dataset may be representative of the population in the Providence area but not necessarily other geographic locations where the model might be used.

Who are the major “stakeholders” in this problem, and what are the consequences of mistakes made by your algorithm?

The most important stakeholder in this problem are patients. Our model will attempt to make classifications based on their image data that will likely be used by physicians or health care professionals to help determine whether a patient should be admitted to a stroke center (which has a higher rate of effectiveness) as opposed to a local clinic. If our model underestimates the severity or even instance of occlusion for a patient, they might not be given the care or treatment that is necessary for them. If our model overestimates the severity of occlusion for some patients, health care professionals may overassign patients not needing the highest amount of care to stroke centers, which could cause higher wait times for patients most in need of care. Another issue, even if our model produces a higher accuracy than physicians, is the extent to which patients feel comfortable having an algorithm make or assist with their medical judgments. For a traditional setting, patients have the luxury of knowing how their physicians arrive at conclusions and ask questions throughout the process, but for an algorithmic one, patients do not have the same degree of transparency and may feel uncomfortable about this.

Other stakeholders in this problem are the medical staff that would be using this technology as well as insurance companies. A physician using this technology as a secondary tool to diagnose and make decisions that impact patients should feel certain that the tool provides a high degree of accuracy. If a mistake is made, who will be held liable for it?

Built With

• python
• tensorflow