We came up with this project as a result of our own hopes and expectations for the future of medicine and informed patient care. As virtual reality and 3D imaging broaden the scope of medicine, we wanted to find an affordable way to convert CT to 3D imaging in order to increase patient education. We believe having a patient learn about surgical procedures in this immersive environment will increase patient understanding and expectations about their procedures, and thus hopefully increase patient satisfaction and overall well-being. Additionally, we hope this would extend to patient health awareness outside of the hospital setting by providing patients a portable and easily understandable way to view their body systems without needing outside knowledge in order to interpret CT scan. Lastly, we wanted to focus on easy accessibility for patients to access these files.
What it does
This process converts 2D CT scans into a 3D model which is then hosted on an 3D image hosting website in which images can be uploaded to virtual reality for an immersive educational experience. Images in this format can also be manipulated by a mouse or trackpad to obtain multiple views of specific areas and improve one's understanding on the anatomy.
How we built it
We extrapolated CT images from DICOM image library into 3D slicer. We edited the compiled images to emphasize specific pathogenic problems in the patients, and the compiled 3D image was uploaded to Sketchfab. On Sketchfab our 3D images can be then viewed in virtual reality because of its compatibility with Samsung Gear and Google Cardboard.
Challenges we ran into
Editing the compiled 3D image in 3D slices was a challenging process, involving our first time experience with the program and the learning curve expected with it. Additionally, due to limitations to hardware available and the power requirements of image analysis, we ran into many delays due to crashing applications. Additionally, we could not use the Oculus CVI as we had hoped due to the graphic card demands.
Accomplishments that we're proud of
We are both confidently and extremely proud in our achievement in accomplishing the goal which we had entered this hackathon with: Converting a 2D CT image into an easily viewable, understandable, portable, affordable, and exciting 3D viewing experience.
What we learned
Frankly, we learned from scratch how to convert 2D images into 3D and also how to then view that image in a virtual space. Additionally, we learned how much open source software is available for many specific and even esoteric uses.
What's next for Heart In Hand
We hope to streamline the process of converting 2D CT images to 3D images, and even expand to other imaging modalities such as Angiograms, Echocardiograms, Ultrasound, and even X-rays. Additionally, we hope to expand upon our mobile application in order for patients to receive their 3D files from their health care facility and easily upload and experience them in the viewer.