MedScape [The Humbugs - ON17]

Keypoint Matching
Aligned Final Product
MedScape WebApp

Inspiration

In the healthcare domain, the significance of medical image registration becomes evident through the compelling statistics and tangible benefits it offers. For instance, the ability to accurately register and compare medical images from different time points or different modalities (like CT, MRI, or ultrasound) can significantly enhance the detection and monitoring of diseases. It can also aid in planning and guiding surgical procedures with greater precision.

As curious university students, we set out to contribute a bit to this immensely interesting world of medical imaging. MedScape is an attempt at diffeomorphic image registration of the retina which helps in getting a unified view of the inner eye by comparing scans from various angles and FOVs.

What it does

MedScape tries to align multiple eye images, volumes, or surfaces to a common coordinate system. This helps us get a unified view of the retina with various imaging modalities and can lead to more accurate diagnoses and better treatment planning. Combining structural and functional images can help identify the exact location and extent of abnormalities, such as tumors, in the eye.

Furthermore, MedScape compare scans of different patients or same patient under different conditions and finds an optimal spatial transformation that best aligns the anatomical structures. We support various types of input images, deformation models, and regions of interest. As a result, MedScape enables clinical applications such as image guidance, segmentation, and reconstruction.

How we built it

The hero of our project is our model which we trained using the FIRE dataset we found on Kaggle. We used a techstack of primarily Python for this project. However MedScape has made use of quite possibly every application of Python. From Neural Networks to Front-end, we made heavy use of Python libraries such as TensorFlow, PyTorch, cv2, Flask and Streamlit.

We used the model structure of SuperRetina for building the convolutional neural network, which was then used for predicting the alignment and testing the model for image registration of retinas; Streamlit was involved in creating a versatile front-end and Flask for tying the whole project together.

Challenges we ran into

First of all, deciding a cause that we were all equally passionate about took days of grueling research and advocating by each and everyone on the team. Secondly, our initially planned model Voxelmorph did not work out in the end moments and we had to switch to our backup SuperRetina, however this came out better than expected as the accuracy we obtained is much better than what we would have gotten with Voxelmorph. Thirdly, Since our project involves a lot of image handling, we assumed Torchvision would be part of the installed libraries in IBM-Z Linux0NE, that was not the case, and hence we had to switch to Tensorflow. This albeit, proved a bit difficult, but in the end yieled the same results that we would have gotten with Torchvision.

These challenges have made this hackathon even more memorable for us, for we learnt to be incredibly versatile while dealing with these obstacles!

Accomplishments that we're proud of

One of our biggest accomplishments during this hackathon project was our research. We researched meticulously about the type of datasets, libraries, and frameworks we want to use. Our expertise in the medical field is quite limited, yet we wanted to create something that would be useful for the field because we were inspired by this topic. Consequently, we ensured to read research papers and expert opinions before starting the project which made our actual coding process easier.

What we learned

During this hackathon, we became familiar with IBMZ and Linux0NE, which allowed us to think of ways we can build a data model. Furthermore, we were all able to gain personal skills such as collaboration, problem-solving, and thinking outside of the box! We used a techstack that included frameworks where all of us were unfamiliar with at least one of the frameworks we were using, but we learned throughout the day which is something we are very proud of.

What's next for MedScape [The Humbugs - ON17]

As for our next step, we hope to improve the accuracy of our images from 78% to be able to find give more reliable information to our user. Moreover, we want to expand the scope of medical image registration past vision and retinas and look at how we can accurately register images of several modalities such as CTs, MRIs and involve it in the imaging of various other body parts too! Lastly, we are think that following the 'tech-for-good' theme, this project truly has the ability to positively impact the medical infrastructure; hence, we want to think of ways we can get funding so we can have a better reach!