Praxis

Home Page
3D Reconstruction
Simulation with Visuals
Exporting

Inspiration

Every year, over 300 million surgeries are performed worldwide, and more than 10% result in errors or reoperations traced back to insufficient familiarity with the patient's anatomy or the procedure itself. Traditional rehearsal methods are disappearing. Cadaver labs are increasingly rare, and high-fidelity simulators cost hundreds of thousands of dollars, leaving most surgeons without a way to practice on anything resembling the actual patient they're about to operate on. This is a space ripe for innovation, and it's where Praxis comes in.

What it does

Praxis lets any medical professional simulate a procedure on a hyper-personalized 3D reconstruction of their specific patient. It starts with the patient's CT scan and surface imaging. Our modeling pipeline generates a full 3D reconstruction built entirely around that individual's anatomy. From there, the surgeon uses natural hand tracking to navigate the model and select what they want to simulate. The orchestration layer generates a live, annotated simulation showing exactly how the procedure plays out on this body: where the risk zones are, what this person's anatomy changes about the approach, and what the surgeon should watch for. An AI surgical assistant narrates context, highlights danger areas, suggests next steps, and generates a post-session summary with risks and contingencies.

How we built it

We built Praxis with a React + TypeScript frontend using Three Fiber for 3D rendering and a FastAPI backend. On the imaging side, we used nibabel and pydicom to ingest NIfTI and DICOM scans, then scikit-image marching cubes to extract anatomical surfaces and export both mesh and Gaussian splat representations. We also used the World Labs API for Gaussian splat generation. For the simulation experience, we used Three.js and MediaPipe for webcam-based hand tracking, letting users point, pinch, trace incision lines, and manipulate the model without any special hardware. On the AI side, we used Kimi K2 Thinking LLM calls with a stateful agent that generates all surgical guidance end-to-end: structured risk analysis, procedure narration, and post-session summaries.

Challenges we ran into

The hardest challenge was accurately going from raw CT scan data to a personalized, usable 3D body reconstruction. Medical imaging data is messy and variable, and building a pipeline that reliably produces interactive reconstructions from both DICOM and NIfTI inputs took significant effort. Beyond reconstruction, making the simulation itself genuinely useful was its own challenge. We had to figure out how to present a simulated procedure in a way that a surgeon can actually read, understand, and learn from in 3D space. Stabilizing hand tracking from noisy webcam input and mapping gestures to meaningful simulation actions added another layer of complexity.

What we learned

None of us had worked with medical imaging data before, especially in a world model context. We learned a ton about navigating and presenting 3D anatomical information in an interactive way. The project pushed us to combine very different disciplines (computer vision for hand tracking, real-time graphics for mesh and 3D reconstruction, and language models for contextual guidance) into a single coherent experience. We came away with a much deeper understanding of how these modalities can come together to create something more powerful than any one of them alone.

What's next for Praxis

Next, we want to integrate Praxis directly into clinical workflows, connecting with existing hospital systems so it becomes part of how surgeons actually prepare for procedures. We also want to refine the AI assistant and reconstruction pipeline to be as anatomically accurate as possible for each patient. We've already spoken with two surgeons who said this fundamentally changes how they think about preparation, especially for procedures they haven't performed in months. Long term, we see Praxis becoming a general platform for personalized surgical rehearsal that plugs into the surgical workflows hospitals already use.