TargetONCO

We are building TargetOnco, the world's first and only fully automated agentic AI system for end-to-end precision oncology radiation & pathological analysis.

By leveraging modern agentic and custom pathological tools, we have created a fully end-to-end pipeline for clinical data processing and deep research — from X-ray radiology to complex spatial proteomics tissue.

Orchestrated by a single AI agent system, we aim to empower oncologists, researchers, and patients in the fight against cancer.

Inspiration

Globally, 1 in 2 people will develop cancer in their lifetime. Every year, 10 million lives are lost to cancer. For patients, the gap between detection, disease, and treatments is weeks of agonizing uncertainty. But this is all our current systems allow.

Why?

Precision oncology is fragmented and manual. Modern microscopy is capable of capturing incredibly high-fidelity OME.TIF data with 40 data channels and millions of unique signals.

Today, a radiologist analyzes X-rays in one silo, while a pathologist analyzes high-fidelity tissue samples (OME.TIF format, 10+ GB per image) in another. Each uses specialized workflows and custom tooling, making this process slow and segmented. In time-sensitive contexts like cancer treatment — where a few weeks changes patient outcomes — this friction is extremely harmful.

We think that these problems are at the core of modern oncology. Solving these issues will save millions of dollars, hours, and, most importantly, lives.

What it Does

TargetOnco is the world’s first autonomous AI agent for precision oncology. It transforms a currently disjointed, multi-week diagnostic process into a unified, real-time conversation.

It is an autonomous oncology analysis platform for patients and doctors.

The system takes in varied clinical data inputs. It then leverages a hierarchical multi-turn agent framework to process this data. Finally, we output:

A deep research medical report
A bespoke chat agent for users to better understand data insights

We have two primary pipelines: ONCO-RAX and ONCOPathology.

ONCO-RAX

Optimized for standard patient X-ray data, it uses a multi-turn agentic system to detect abnormalities. By combining statistical inference with the Elasticsearch API, it transforms raw imagery into structured, actionable clinical reports.

ONCOPathology

Built for high-fidelity OME.TIF data, this pipeline supports advanced clinical research across 40 input channels. It performs intensive data processing — including TMA arraying, cell probability mapping, watershed segmentation, and advanced quantification — to identify complex protein-biomarker interactions for analysis.

Because accessibility is a core pillar of our mission, we’ve also integrated AgentChat. This interface allows both patients and doctors to navigate complex data through natural dialogue.

To make this experience truly personalized, TargetONCO uses biomarker matching to identify doppelgangers — past patients with near-identical tumor profiles. This allows us to offer high-confidence, data-driven insights like:

Based on 5,000+ similar cases, this patient has an 85% probability of responding to Immunotherapy X.

How We Built It

The frontend is built with React 19, TypeScript, and Vite, tailored for clinical precision. We integrated xterm.js to render a real-time terminal, giving researchers full visibility into the agent's thought process and tool execution logs. Data is streamed via FastAPI using WebSockets so when an agent discovers an insight, it is visualized on the dashboard.
The brain of our system is a multi-turn agent powered by the Claude Agent SDK. Unlike standard chatbots, this agent maintains a persistent, context-aware session capable of hierarchical planning. It orchestrates the entire workflow — from parsing user intent to executing complex bio-tools — enabling the system to reason through clinical data.
We integrated a dual-pipeline approach for clinical information processing:
- OncoRAX: A specialized pipeline that processes chest X-rays using deep learning. It uses a Critique-Retrieval-Refine loop to detect malignancies with ~95% accuracy on benchmark datasets.
- OncoPathology (Proteomics): A spatial proteomics workflow that handles raw OME-TIFF images. It automates segmentation, quantification, and phenotyping using a suite of containerized tools (UnMicst, S3segmenter).
We utilized the Modal SDK to manage data processing. Modal allowed us to offload resource-intensive tasks in our dense data (e.g., probabilistic map generation and cell segmentation) into cloud sandbox environments.
We implemented Elastic (Elasticsearch) to power our vector search engine. By generating 70-dimensional tissue embeddings and 10-dimensional cell embeddings, researchers can easily find similar patient profiles and biomarkers.

What We Are Proud Of

A traditional oncology process has multiple clinicians and days-to-weeks of turnaround. TargetOnco turns this into minutes of autonomous inference at approximately $1 of API cost.
- TargetONCO runs the complete oncology pathological analysis pipeline, from raw OME-TIFF to clinical insights, within 10 minutes. This same workflow currently takes weeks of manual effort across specialists and systems in clinical and research settings.
Users are not locked into a fixed workflow. Every stage is independently callable, so TargetONCO can adapt to where a hospital or lab already is in their workflow.
We believe TargetONCO is the first system to perform tissue-level patient history matching using actual molecular profiles. Current approaches to patient matching are naive (pairing patients by age, tumor site, or staging).

Our system:

Takes a patient’s raw OME-TIFF
Quantifies every cell’s protein expression
Computes per-marker distributional statistics
Compresses it into a 70-dimensional tissue embedding

We then use Elasticsearch to search across extensive Orion-CRC colorectal cancer patients.

Challenges

Converting MedRAX into a multi-turn reasoning system was a major hurdle. We had to completely redesign the execution flow to support complex features like self-critique and data retrieval without destabilizing the core inference logic.
Deploying Elasticsearch for hybrid search was difficult. To get accurate results, we had to fuse traditional keyword scores (BM25) with semantic vector scores while keeping data ingestion under rate limits.
The raw outputs from the model were not immediately clinically useful, so we implemented Bayesian calibration. We needed to estimate priors from the dataset to ensure probability rankings were meaningful for medical decisions.
Handling high-dimensional Jina embeddings required robust fault tolerance. We implemented exponential backoff logic to prevent failures when dealing with large data volumes.

What’s Next

We believe the future of clinical healthcare is not just about accumulating more data, but about giving that data a voice to speak directly to clinicians.

We see systems like these as the blueprint for how humanity will confront disease in the coming decades: agents, compute, and tool-driven discoveries.

Expanded Disease Analysis

Our focus for Onco was to tackle cancer — one of humanity’s most prolific health challenges. However, disease prevention across conditions can suffer from the same structural problems.

We believe it is possible to extend the deep domain expertise we built for oncology and adapt it for other conditions.

Our vision is for Onco to become the universal platform for agentic medical solutions.

High-Performance Computing

High-fidelity biology requires high-performance computing. We envision a future where the infrastructure is invisible to the clinician.

Our next step is to empower our agents to assess the computational weight of requests and dynamically allocate to HPC clusters.

Improved World Knowledge

In oncology, critical breakthroughs often live in preprints and clinical trial logs long before they reach textbooks.

We plan to implement continuous, autonomous crawlers that ingest real-time data from bioRxiv and clinical databases.

For an extended architecture breakdown and discussion, please check out our GitHub. We really loved working on this project, and hope you do too: https://github.com/tomtommyyuan/targetONCO