OncoVision

Inspiration

Praneeth's grandpa passed away from brain cancer and throughout process he saw the stress in his parent's eyes dealing with the inconsistent timelines, seeing how he was degrading, how what the doctors explained to his family didn't feel grounded in reality.

Chaandan is a quantitative biologist that did research at a company in Fremont, Praneeth is researching explainability for AI at the Department of Motor Vehicles, Vinesh works in software engineering at the Department of Conservation, and Nisheeth is a Hardware Lead at Slugbotics. Since Praneeth and Vinesh work in the state government, involving alot of sensitive data and documents, all of their workflows revolve around secure local genAI deployments.

What it does

OncoVision simluates how a glioma could grow in 3D, starting from a patient's real MRI tumor outtline. You can compare treatment scenarios side by side. A local language model explains each result live, checking against actual numbers with verification suites. Everything runs locally on the GX10.

How we built it

We polled data from 676 patients from 3 public glioma datasets (UCSF, LPTDG, etc) plus LUMIERE for growth modeling. The simulation is a 3d reaction-diffusion tumor model in PyTorch batched on the GB10 GPU and calibrated to each patient's own scans. XGBoost for mortality risk and growth rate and SHAP explaining which features drive each score. Nemotron with 2 agents (draft + audit), and a judge that verifies every number and direction. A fully offline Three.js 3D viewer is used for viewing

Challenges we ran into

Our first challenge was harmonizing data across institutions as each cohort used different tools, schedules, and definitions. Pooling them together was pretty ambitious as we had to make a single shared contract.

Accomplishments that we're proud of

We scaled from 20 patients on a MacBook to 676 across three hospitals. Runs fully on the GX10, and we fit our 28gb model, simulation, and cohort into a 128gb memory pool. We built a argumentative model architecture (advocate, skeptic, judge) verifying our explanations before they reach the patient.

What we learned

We learned that memory is a super big factor in speed, as the GPU sped up simulation by 2.3x compared to our prior iteration on an M4, but the shared memory lets the model, simulation, and data run together at blazing speeds. Another thing we learned is that models that look good in one hospital can fail in another, as our model initially scored .78 AUC within UCSF, but much lower on unseen cohorts.

What's next for oncovision

Use the GPU for simulation ensembles to show uncertainty, and definitely bring in data with dated doses and response tracking, so drug effects can be measured instead of assumed. Add an independent AI auditor. The most ambitious thing we would love to implement is applying the same local verified pipeline to other cancers where imaging privacy rules out cloud usage

Built With

fastapi
nibabel
numpy
pandas
pyarrow
pytorch
scikit-learn
scipy
three.js
xgboost

Submitted to

UCSC AI Hackathon

Created by

I trained the glioma risk model and designed the multi-agent explanation system at the heart of OncoVision: an Advocate that argues for a finding, a Skeptic that challenges it, and a Judge that rules on the evidence, so every explanation shown to users has been checked before it appears.

Praneeth Mandalapu
I architected OncoVision's completely air-gapped frontend, ensuring the medical viewer ran entirely offline to guarantee zero latency and strict HIPAA compliance. To achieve this on the ASUS GX10, I engineered a highly optimized binary data pipeline using modular vanilla JS, shrinking a 95MB monolithic payload into lightweight structures that smoothly render 3D tissue scans without maxing out local memory.

Nisheeth Velicheti
I focused on the data engineering and medical imaging side of OncoVision. My work involved preparing brain MRI segmentation masks, integrating clinical data from multiple glioma datasets, and supporting quality checks to ensure consistent tumor measurements. I helped connect the imaging data to our 3D tumor growth simulation and AI prediction pipeline. I also contributed to preparing our hackathon demo and communicating the project’s capabilities and limitations. It was a great opportunity to apply programming and data science to a real-world healthcare research problem.

CDanTheMan
Vinesh Balamurugan