3D Medical Image Viewer Pack

Inspiration

Medical professionals rely on sharing 3D medical images to care for their patients; sharing medical images lets these professionals get a colleague's second opinion on a medical case and publish their latest breakthroughs in their research. However, it is difficult to make 3D medical images adequately interactive.

Currently, medical professionals are positioned between two extremes when they need to share a 3D medical image; they can either send a colleague a screenshot/screen recording or the entire image file. Screenshots and screen recordings are not very interactive. With these formats, recipients can only view the portion of the 3D image that the sender decided to show. On the other hand, it can be excessive to send an entire image file. People need specialized software to open 3D images, and they might not want to navigate away from their current work to open a different application.

This project aimed to make a tool that provides a balance between those extremes.

What it does

The "3D Medical Image Viewer Pack" provides a formula to represent a Nifti image, a commonly used filetype for 3D medical images, as an image slider. By moving the slider left and right, you can see different 2D frames of a 3D image along the axial plane. When the slider is placed all the way to the left, you are seeing the bottom-most 2D frame on the axial plane; when the slider is placed all the way to the right, you are seeing the top-most 2D frame. Upon calling this pack's formula, it will automatically insert the image slider into the document.

The formula is easy to use. Just call CreateNiftiViewer and pass the URL to your Nifti file as a parameter.

This pack combines the minimalism of a Coda document with the interactivity of a 3D image. In some situations, it can be much easier to share a Coda document with an image slider than to share an entire medical image file.

How we built it

We built the formula using Coda's pack development library. The formula embeds an image slider that is externally hosted on a website. This slider was built with HTML, CSS, nifti-reader-js, and javascript.

Coda Pack: https://github.com/jrmkim50/coda-nifti-pack
Nifti Image Slider: https://github.com/jrmkim50/js-nifti-viewer
Proxy Server: https://github.com/jrmkim50/cors-proxy-express-server (we proxied the image download through our own server hosted on Heroku)

Challenges we ran into

We had a lot of difficulty reading the Nifti file. Originally, we experimented with directly embedding the Nifti file into the Coda document with the "/embed" command. However, we had a lot of difficulty reading the embedded file from a Coda pack, so we went with having the Nifti file hosted outside of Coda. Our Coda pack will download the externally hosted file and then display the image. We also had difficulty showing an image slider. We solved this issue by using Coda's content embedding capabilities.

Accomplishments that we're proud of

We had not made a Coda pack before this hackathon. We are very proud to have made our first Coda pack. We are also proud of our pack's capabilities. To our knowledge, our pack is one of the first plugins on any document viewer to let viewers interact with medical images with 3D sliders.