DeepBrainSeg

Inspiration

• Brain Tumors grow in various shape and texture, resulting in huge stochastic predictions by radiologists, automated tools using deep learning can help to reduce false positives in radiologists predictions

• In the hospitals with an inefficient workflow managements, tools like these will help radiologists to analyze critical cases at the first

• Radiomic features from the tumor region provide lots of information about the tumor and help ineffective treatment, which can be easily extracted with such automated tools

What it does

• The accurate automatic segmentation of gliomas and its intra-tumoral structures is important not only for treatment planning but also for follow-up evaluations. Several methods based on 2D and 3D Deep Neural Networks (DNN) have been developed to segment brain tumors and to classify different categories of tumors from different MRI modalities. This project utilizes an ensemble of the fully convolutional neural networks (CNN) for segmentation of gliomas and its constituents from MRI.

How I built it

• The application was developed using various opensource tools, the details can be found in the repository readme, here we made use of PyTorch the process of development involved various iterations, PyTorch provided the ability for quick prototyping and development of deep learning models.

• The data loader and data augmentation tools provided by torch and torchvision were used in training pipeline, the easy integration of these tools with our pipeline helped us in the developing better tool for tumor segmentation.

• The proposed ensemble comprises of 3 networks, two 3-D and one 2-D network. Of the 3 networks, 2 of them (one 2-D & one 3-D) utilize dense connectivity patterns while the other 3-D network makes use of the residual connection.

• Additionally, a 2-D fully convolutional semantic segmentation network was trained to distinguish between air, brain, and lesion in the slice and thereby localize the lesion in the volume. Lesion localized by the above network is multiplied with the segmentation mask generated by the ensemble to reduce false positives.

• With the proposed technique we achieve the dice of 0.89 0.76, 0.76 on the whole tumor, tumor core and active tumor respectively. In this, we also provide the technique for uncertainty estimation, both model and data uncertainty which helps radiologists in making better decisions.

Challenges I ran into

• Architecture development: In this project fixing on the types of network architecture was a huge challenge, it was fixed based on the literature survey and the availability of resources. The networks used were finally of 57 and 23 layers of CNN

• Hyperparameter selection: selecting parameters for data preparation, a learning rate scheduler for training, batch size, and input data dimension was kind of huge challenge

• Loss function selection: After various experiments, the weighted combination of cross-entropy with dice loss was considered for error calculation, the weights for the same were estimated experimentally

• UI development: Developing python-based stand-alone UI, took lots of iterations

• Packaging: We make use ANTs for skull stripping the brain volumes, scripting to install all the pre-requisites supporting various platforms and various python versions, consumed lots of time and involved lots of iterations

Accomplishments that I'm proud of

• The final pip package and UI: After the development, it was tested with the radiologist, the reaction of the radiologist towards this application made me feel proud about myself

What I learned

• Understanding actual problems in medicine and radiology

• Developing products satisfying the end-user objective

• Steps involved in solving real-life problems, in terms of research and implementations

What's next for DeepBrainTumorSegmentor

• Radiomic feature extraction pipeline

• Explainability of these models

Built With

• nibabel
• pytorch
• tkinter