SurgiPlanXR

• 

  SurgiPlanXR

• 

• 

• 

• 

• 

• 

• 

Inspiration

Pre-surgical planning is a 3D problem currently trapped on 2D screens. Surgeons have to mentally reconstruct flat CT scans, which increases cognitive load. While hardware like the Phantom Omni allows for 3D haptic medical planning, these setups cost upwards of 3000€ and are tethered to specialized workstations.

When I saw the Logitech MX Ink track, I realized that consumer XR combined with the precision of a tracked stylus could be a good fit spatial implant design. The goal of SurgiPlanXR is to replace an expensive setup with an accessible, highly precise XR workflow where a surgeon can naturally draw, design and "feel" patient anatomy.

What it does

SurgiPlanXR is an XR spatial planning tool that allows medical professionals to interact directly with patient anatomy (like a 3D skull model) projected right onto their real-world desk using passthrough.

Instead of using a clunky joystick, users leverage a stylus workflow to:

• Directly Annotate: Tap points precisely along the surface of the bone to outline a cranial defect.
• Generate Implants: With the press of a button, the app instantly generates a custom, 1mm-thick 3D mesh (representing a titanium implant).
• Spatial Manipulation: Grab, cut and move the bones to prepare for the implant.

How we built it

The prototype was developed in Unity using the Meta XR SDK. The core of the application relies on raycasting and surface normal alignment. When the stylus tip touches the bone, the app calculates the hit.normal to ensure anchor points sit on the surface.

For the generative design, I wrote a custom procedural mesh generation script. It takes the user's drawn points, calculates a centroid, and triangulates a dual-layer mesh (extruding outward along the normal by exactly 1mm) to instantly create a volumetric solid rather than a flat 2D sticker.

Challenges we ran into

The hardest part was the procedural mesh generation and aligning it to the skull. Another challenge was simulating the "slicing" of bone. I solved it with OpenFracture for now, but when dealing with real surgical resection a more advanced slicing will be required.

Accomplishments that we're proud of

I am incredibly proud of getting the real-time procedural mesh generation working. Seeing a custom, 3D "titanium" plate materialize where I drew it is a powerful "wow" moment. It validates the entire concept that generative design can happen spatially, at the tip of a pen, rather than requiring complex CAD software.

What we learned

I learned the massive difference in user experience between standard VR controllers and precision tools. Building this made it obvious that for medical tooling, standard controllers are just too blunt.

What's next for SurgiPlanXR

This Phase 1 submission is just the foundation. If selected for the Semi-Finals and equipped with the MX Ink hardware, the immediate next steps are:

• Haptics: Integrating the MX Ink's high-frequency actuators to provide tactile feedback so the surgeon "feels" the bone density when the stylus makes contact.
• Better slicing: Adopting a more advanced slicing method, will enable surgeons, to do real planning, simulation and training, utilizing SurgiPlanXr and the MX Ink stylus. If in scopte, implementing true CSG boolean operations so the stylus can carve away bone dynamically, would be optimal.
• STL Export: Adding a feature to export the generated implant mesh so it can be sent directly to a 3D printer for physical manufacturing.

Built With

• c#
• unity