Alpha Tau Submission

Inspiration

The approach was inspired from 3D Instance Segmentation but as 3D Instance Segmentation could not be implemented within the timeframe a 2 step strategy of 3D Semantic Segmentation and 2D Object Detection was adopted.

What it does

The Pipeline Proposed locates SEEDS present in the CT-SCAN image and gives their location and orientation as output.

How we built it

The project was built using the libraries Pytorch, Monio, Numpy, Pandas, Matplotlib

Challenges we ran into

The 3D dataset was huge and the seeds occupied a very small fraction of the total volume. Instance Segmentation in 3D is not mainstream yet so alternative approaches had to be adopted. Semantic Segmentation needs a lot of time to train to give satisfactory result. As models other than Cascade RCNN did not produce results under limited training time WBF could not be applied.

Accomplishments that we're proud of

The best Object detection model(Cascade RCNN in our case) was successfully able to predict the location of a few SEEDS and return their positions. GIven enough time to train the entire pipeline should work well with the proposed Problem.

What we learned

3D Instance Segmentation model is really hard to train in this case as the Ratio of the volume occupied by the seeds to the total volume of the CT-SCAN is very less. Due to the extremely small size of the SEEDS most object detection models will fail unless trained for a really long time.

What's next for Alpha Tau Submission

Training models other than Cascade RCNN for larger number of iterations so that the ensemble can be better. Using more architectures in case of 3D segmentation model so that a better segmentation mask can be obtained. Using 3D instance segmentation models to detect and separate seeds in one shot. 3D ROI Pooling could be tried as an approach to localization.