Alpha DaRT seed detection from CT images

Inspiration

Treating solid tumors with alpha-emitting radionuclides is a game-changer development in radiotherapy. The invasive procedure of seed implantation inside the solid tumor requires high accuracy, precision, and skills. However, there is no efficient way to map the location of the DaRT seeds within the body to ensure that sufficient tumor coverage has been achieved. Computer vision tools can potentially be used to address this challenge.

What it does

Digitization of alpha-emitting seeds over the CT images is a challenging task due to its miniature sizes and interobserver variability. Machine/Deep learning-based detection and segmentation of seeds can play an important role in minimizing the interobserver variations which will lead towards the more accurate dose estimation of the tumor and nearby healthy tissues.

How we built it

We tried to pre-process the image and mask dataset, but that could have resulted in data loss as seed dimensions are in millimeters. We have used U-Net architecture using tensorflow platform. We firstly studied the sample code and structure from the workshop and built on top, as a quick and fully structured solution. The algorithm we use is for 2D images, so we generated 2D images from 3D models. The process to achieve this must be in two steps, firstly find seed from the image and mask it, then calculate the vector information based on the first mask showing the seed and the last mask before the seed disappeared. We achieved the first step, our program can recognize the seed from analyzing the images. We didn’t achieve the second step because of the time limit. We also tried 3D U-Net architecture to identify the seeds but due to the inclusion of multiple libraries we are unfortunately unable to implement the same in a short span of time.

Challenges we ran into

Installing packages from multiple sources, different frameworks, cloud-based environments, and efficient use of computation resources are various challenges for us. Apart from this, preparing a dataset for the model input is also a challenge, and choosing the best among the many is still getting to know. What’s more, we need to share knowledge with each other simply and clearly to achieve the program as much as we can, which is not easy because picking any of the topics from this program can be difficult.

Accomplishments that we're proud of

We have learned the basics of image processing, 3D data handling, and visualization of 3D image datasets. We have networked with people from different backgrounds and skills. We finished the base functions and process in 48hrs, and implement the 3D visualization for the seeds.

What we learned

We learned a lot about bringing together the different disciplines of physics, image feature analysis, machine learning, and the challenges that pose. Additionally, we learned that simple applications to a problem can become much more complicated very quickly when met with the realities of variation in images and seed placement in actual CTs. The more “hard cases” encountered the more challenging it becomes to adapt the algorithm to the new difficulties.

What's next for Alpha DaRT seed detection from CT images

Although simple detection of seeds from individual CTs can be programmed, our rigorous approach is still needed to evaluate seeds at orientations nearly parallel with the CT slices with minimal representation in slices, multiple seeds placed very close to each other, CT images taken from different CT scanners having different CT to HU calibration Curve, etc. Further algorithms will need to be developed in order to efficiently and ably assess these cases, perhaps by the introduction of special sub-algorithms that evaluate differently shaped densities within CTs (either by specific searches or additional machine learning approaches).