Enhancing Hemodynamics Simulation with XR Integration

Comparison between physical simulador and the XR layer
Physical simulator during a workshop of the procedures
Angiographer image (the green x-ray tube is out of view)

Inspiration

I was developing a realistic 3D-printed hemodynamic simulator with tactile feedback, in collaboration with a surgeon. We added cameras to track the catheter, but in actual surgery, the surgeon only sees the angiography image. Though our model is radiopaque, using real equipment is too costly.

What it does

He explains the experience, then adds an angiography simulator to the user’s view, showing a standard Mitral Valve-in-Valve and transseptal puncture procedure. Preserving the physical simulator’s camera views and tactile feedback from the physical simulator but hidding it and only shouing the image.

How we built it

The project was developed in Unity using the fo-dicom plugin to perform DICOM volume rendering and retrieve the slice relative to the angiographer’s position and rotation. We already the demo exam, used as a reference for the physical simulator development.

Challenges we ran into

I couldn’t make the fo-dicom plugin work on the standalone Oculus. So it is a PCVR experience. A future technical challenge is adding catheter position tracking—possibly through sensor integration.

Accomplishments that we're proud of

I’m proud to have understood the fo-dicom plugin and added something initially outside the simulator’s scope. Now the team is eager to push it further.