OncoTwin 3D — DataHub Cancer Context Mission Control

Seven cancer-context digital twins where DataHub-grounded agents detect unsafe ML conditions, block consumers, repair governed data and write reusable knowledge back.

Comment

Inspiration

Cancer research ML pipelines don't fail loudly — they fail quietly. A single-cell matrix goes stale, a biomarker column picks up a run of silent NULLs, a schema mutates upstream, or a provenance link between genomic, protein and spatial evidence just... disappears. None of that shows up as an error. It shows up three hops downstream as a wrong progression score on a dashboard someone trusts. The inspiration for OncoTwin 3D came from a simple frustration: an LLM agent looking only at table names and a prompt has no way to know any of this is happening. It doesn't know what's upstream, what's downstream, who owns the data, whether there's already an open incident, or whether a "fix" it proposes is even schema-compatible. What agents actually need to act safely on regulated, high-stakes data is the thing a metadata catalog like DataHub already holds — canonical identity, lineage, ownership, quality history, and governance state. We wanted to prove that context out loud, in a form a judge (or an oncology data platform lead) could see rather than take on faith — hence a 3D digital twin instead of a log file.

What it does

OncoTwin 3D is a DataHub-native safety control plane for cancer research ML, wrapped in a Three.js "cancer context universe." A judge can trigger one of seven synthetic reliability failures spanning cohort, single-cell, genomic, protein and spatial data products — for example, the flagship Biomarker Completeness Crisis. When a failure is triggered:

  1. Six collaborating agents pull live evidence from DataHub via a self-hosted MCP Server and the Agent Context Kit — canonical URN, schema, ownership, lineage, generating queries, and any active incidents.
  2. ML Guardian turns that context into a digital-twin risk state and blocks the downstream ML consumer (progression_scores) before it can serve bad predictions — proving the blast radius via real lineage, not a guess.
  3. Repair Engineer generates catalog-compatible BigQuery SQL (COALESCE-protected) from the actual schema and contract.
  4. The system pauses for a human approval secret — nothing mutates without it.
  5. On approval, BigQuery repairs progression_features, regenerates progression_scores, and validates zero-NULL signal.
  6. Governance Steward resolves the real DataHub incident, writes an AgentRepaired tag, and persists responsible agent, timestamp, job IDs, validation status and a SHA-256 receipt back into DataHub properties.
  7. A final MCP read-after-write proves the next human or agent actually inherits that knowledge. All seven missions have their own dedicated BigQuery table, DataHub URN, owner, contract, and lineage, and each is rendered as a distinct 3D visual world (biomarker constellation, drift wavefront, schema rift, protein folding, spatial immune escape, etc.), with a Proof Galaxy that lets a judge capture a live, timestamped, exportable JSON receipt showing 6/6 real MCP reads and mutation_performed: false until approval is given.

How we built it

Context layer: DataHub Core running as Docker containers on a Compute Engine VM, ingesting BigQuery schemas, profiles, usage, queries and fine-grained lineage; a self-hosted DataHub MCP Server plus the Agent Context Kit expose search, entity, schema, lineage and query tools to the agents. Agent layer: A six-agent theatre (Context Scout, Lineage Sentinel, ML Guardian, Bioinformatics Agent, Repair Engineer, Governance Steward) built with LangChain + Vertex AI Gemini, following the same search → quality → lineage → repair pattern as the official DataHub Skills. Safety policy: A deterministic tabular Q-learning controller converts the retrieved DataHub context (data trust, model risk, null rate, drift, schema compatibility, consumer-blocked state) into a safety action — block, quarantine, freeze, flag, or repair — never a treatment recommendation. Application layer: A FastAPI "Mission Control" backend on Cloud Run orchestrates missions, streams the agent trace over Server-Sent Events, enforces the human-approval gate, and talks to BigQuery for the actual repair/validation jobs. Governed writeback: The official DataHub REST emitter writes durable descriptions, AgentRepaired tags, and custom properties (agent, timestamp, job IDs, validation result, receipt hash) back into the catalog. 3D twin: Three.js renders locally-vendored anatomical GLB meshes (lung/heart/liver/kidney) with selectable lesions, internal scRNA layers, and a five-stage progression playback, all wired to a DataHub-to-scene event bridge so every MCP call and governance action visibly moves the scene. Infra/security: Secret Manager holds tokens and the approval secret; Cloud Run reaches DataHub GMS over a private VPC, with judges viewing the DataHub UI itself through an IAP tunnel — credentials never touch the browser.

How to interact with Onco3D

LIVE APPLICATION

https://oncotwin-mission-control-wslhcl5ziq-el.a.run.app

OncoTwin 3D is designed for a desktop browser. For the best experience, open the application full-screen in Chrome, Safari or Edge.

QUICK DEMO — 3 TO 5 MINUTES

  1. Open the Live Mission tab.

  2. Select “Biomarker Completeness Crisis · LIVE DATAHUB.”

  3. Click “Run selected mission.”

  4. Follow the six-agent execution:

• Context Scout discovers the canonical DataHub asset. • Lineage Sentinel identifies upstream and downstream dependencies. • ML Guardian evaluates whether model consumption is safe. • Bioinformatics Agent interprets schema-grounded biological context. • Repair Engineer generates a metadata-aware BigQuery repair. • Knowledge Steward prepares governed DataHub writeback.

  1. Click an agent in the live context graph to inspect:

• The DataHub capability it used • The exact asset URN • Metadata returned by DataHub • Its decision and contribution • Execution timing

  1. When “Approval Required” appears, reviewers with the privately supplied demonstration secret can enter it and click “Approve mission.”

  2. Observe the governed recovery:

• Repair execution • Data-quality validation • Incident resolution • Model-consumption unblock • Agent knowledge written back to DataHub • Final auditable receipt

No public visitor needs the approval secret to explore the read-only DataHub evidence, visualizations and verified replays.

EXPLORE THE APPLICATION

LIVE MISSION

Run one of seven cancer-context safety missions:

  1. Biomarker Completeness Crisis
  2. Tumour-State Progression Surge
  3. Cancer Cohort Drift
  4. Genomic Schema Mutation
  5. Multi-omic Biomarker Discordance
  6. Protein Conformation Evidence Rift
  7. Tumour Microenvironment Escape

The biological failure signals are safely simulated using synthetic or de-identified research concepts. The associated schemas, URNs, queries, lineage and metadata evidence are retrieved from the live DataHub deployment.

CANCER TWIN

Explore the interactive Three.js anatomical twin.

• Select a cancer cohort. • Drag to rotate the organ. • Scroll to zoom. • Click a cancer marker to inspect it. • Change progression stage. • Play the progression animation. • Compare the biological state with DataHub trust and ML risk.

This visualization is for research demonstration only and does not provide diagnosis or treatment advice.

scRNA

Explore the single-cell RNA view.

• Hover over cell clusters. • Inspect epithelial, immune, myeloid and malignant populations. • Change gene programs such as MKI67 and EPCAM. • Review DataHub-backed schema evidence. • See how governed metadata gives meaning to each cell-state visualization.

CAUSAL OBSERVATORY / CONTEXT GRAPH

Inspect the metadata and ML dependency graph.

• Follow raw and transformed datasets. • Explore upstream and downstream lineage. • Observe how one degraded feature dataset affects downstream scoring. • See why the ML Guardian blocks or permits model consumption. • Inspect the owner, schema, quality and query evidence supporting each decision.

PROOF GALAXY

Proof Galaxy is the recommended judge-verification experience.

  1. Open the Proof Galaxy tab.
  2. Select one of the seven 3D mission worlds.
  3. Click “Capture fresh proof again.”
  4. Watch the application execute six live DataHub MCP reads:

• search • get_entities • list_schema_fields • get_lineage — upstream • get_lineage — downstream • get_dataset_queries

  1. Inspect the exact DataHub URN, response timing and result for each call.
  2. Copy or record the generated SHA-256 proof receipt.

All seven conditions have been independently verified with 6/6 live DataHub MCP reads, producing 42 condition-specific reads in a complete verification run.

GENERATED FIX

Open the Generated Fix tab to inspect:

• The original failure • DataHub schema evidence • Root-cause explanation • Agent-generated BigQuery SQL • Downstream impact • Human approval state • Validation outcome • DataHub writeback result

The repair is generated from catalogued schemas and lineage rather than guessed table or column names.

VERIFIED REPLAY

After completing a live mission, select “Verified Replay” to replay the recorded agent events without performing another mutation.

Replay mode is safe for demonstrations: it shows the evidence, decisions and recovery sequence while preventing duplicate write actions.

WHAT IS LIVE AND WHAT IS SIMULATED?

Live:

• Self-hosted DataHub on Google Cloud • DataHub MCP Server calls • BigQuery metadata ingestion • Asset discovery and canonical URNs • Schema inspection • Upstream and downstream lineage • Query evidence • Proof receipts • Human approval gate • Governed metadata and incident writeback

Simulated or research-only:

• Cancer failure triggers • Patient-independent biological states • Progression scenarios • RL safety state transitions • Molecular hypotheses

No personally identifiable patient information is used.

RECOMMENDED JURY STORY

For the strongest read-and-write demonstration, run “Biomarker Completeness Crisis · LIVE DATAHUB.”

The mission raises a DataHub incident on progression_features, discovers progression_scores downstream, blocks unsafe ML consumption, generates schema-grounded repair SQL, waits for human approval, validates the repair, resolves the incident and contributes knowledge back to DataHub.

DEMONSTRATION ACCESS

The public application can be explored without credentials.

The approval secret is intentionally not stored in GitHub or exposed in the browser. It will be supplied privately to authorized reviewers who need to test governed writeback.

Direct access to the self-hosted DataHub administration interface is protected through Google Cloud IAP and is not publicly exposed. DataHub evidence can still be inspected safely through Proof Galaxy, the agent inspector and the context graph.

Accessing the app

Open the desktop application and start with Live Mission → Biomarker Completeness Crisis → Run selected mission. Click each agent to inspect its DataHub evidence, then visit Proof Galaxy and click “Capture fresh proof again” to execute six live MCP reads and generate a unique proof receipt. Explore Cancer Twin for interactive 3D progression, scRNA for governed cell states, Context Graph for lineage and Generated Fix for the schema-grounded repair. Public exploration is read-only; the governed writeback approval secret is supplied privately to authorized judges.

Dathub usage

How We Used DataHub

DataHub is not a decorative catalog in OncoTwin 3D—it is the context and governance foundation that allows the agents to act safely on cancer-research data.

1. Self-hosted DataHub Core

We deployed DataHub Core v1.7.0 on a Google Cloud VM using Docker. The deployment includes DataHub GMS, the frontend, MySQL, Kafka and OpenSearch.

The DataHub API is accessible only to the application through the private Google Cloud network. Administrative access is protected through Google Cloud IAP rather than publicly exposing the DataHub server.

2. BigQuery metadata ingestion

Our de-identified cancer-research datasets are stored in BigQuery and ingested into DataHub.

The ingestion captures:

  • Dataset and table identities
  • Column names and data types
  • Dataset profiles
  • Query history
  • Usage metadata
  • Table-level lineage
  • Fine-grained column lineage
  • Container and platform relationships

Our verified ingestion run discovered 12 BigQuery tables and produced 1,227 DataHub metadata events without failures or warnings. It also catalogued 208 query entities and lineage across the cancer-context datasets.

The catalogued assets include:

  • progression_features
  • progression_scores
  • gene_expression_summary
  • tumour_state_transitions
  • cohort_drift_metrics
  • genomic_schema_contract_events
  • multi_omic_biomarker_evidence
  • protein_conformation_states
  • spatial_microenvironment_states

3. DataHub MCP Server

The agents access DataHub through its self-hosted MCP Server instead of guessing schemas, table names or relationships.

Each cancer-context mission performs six condition-specific MCP operations:

  1. search — discovers the canonical DataHub asset.
  2. get_entities — retrieves the asset identity and metadata.
  3. list_schema_fields — reads the real schema contract.
  4. get_lineage upstream — identifies data provenance.
  5. get_lineage downstream — calculates the blast radius.
  6. get_dataset_queries — retrieves available query evidence.

All seven missions have been independently verified with 6/6 successful MCP reads, resulting in 42 live DataHub operations during a complete verification run.

Every mission produces a fresh SHA-256 proof receipt containing its asset URN, MCP operations, timings and evidence. This allows judges to distinguish live DataHub interaction from prerecorded or hard-coded output.

4. DataHub Context Graph and lineage

DataHub lineage gives the agents an end-to-end understanding of how cancer-research data moves through the system.

For example:

gene_expression_summary
        ↓
progression_features
        ↓
progression_scores
        ↓
ML consumption

When progression_features becomes unhealthy, DataHub shows that progression_scores is a downstream consumer. The ML Guardian uses this context to block model consumption before unreliable features reach the scoring workflow.

The application presents this evidence through an interactive context graph and the 3D Proof Galaxy.

5. Schema-grounded code generation

The Repair Engineer reads the real source and target schemas from DataHub before generating BigQuery SQL.

For the Biomarker Completeness Crisis, it discovers fields such as:

  • patient_key
  • cluster_id
  • stage
  • gene
  • mean_expression
  • proliferation_signal
  • epithelial_signal
  • mesenchymal_signal

It then generates a repair using the actual catalogued columns rather than hallucinated names. Missing biomarker observations are safely handled using COALESCE, followed by quality validation and downstream score regeneration.

This demonstrates metadata-aware code generation grounded in DataHub schemas, lineage and query evidence.

6. DataHub Skills

We installed the official DataHub Skills workflow for Gemini CLI, including:

  • datahub-search
  • datahub-lineage
  • datahub-quality
  • datahub-enrich
  • datahub-setup
  • using-datahub

The Context Scout, Lineage Sentinel and Repair Engineer use these catalog workflows to search assets, inspect lineage, assess quality and prepare metadata-aware repairs.

The Skills installation is reproducible on the Google Cloud VM and is documented in the repository.

7. Agent Context Kit and LangChain

The ML Guardian uses DataHub-grounded context as part of its decision process. Data trust, lineage impact, schema compatibility, active incidents and downstream dependencies are converted into a structured safety state.

This context is supplied to the agent workflow through the Agent Context Kit/LangChain integration rather than relying solely on the language model’s internal knowledge.

The model therefore reasons over organizational metadata retrieved from DataHub.

8. DataHub incidents and health governance

The Biomarker Completeness mission demonstrates a complete governed incident lifecycle:

  1. An incident is raised against progression_features.
  2. DataHub marks the asset as unhealthy.
  3. Upstream and downstream lineage are inspected.
  4. The ML Guardian blocks downstream model consumption.
  5. The Repair Engineer generates schema-grounded SQL.
  6. The workflow pauses for human approval.
  7. The approved repair executes in BigQuery.
  8. Data quality is validated.
  9. The DataHub incident is resolved.
  10. Model consumption is unblocked.

This makes DataHub part of the operational control loop rather than only a passive metadata store.

9. Governed DataHub writeback

Write operations are protected by a human approval gate. The approval secret is stored in Google Cloud Secret Manager and is never exposed in the browser or committed to GitHub.

After approval, the agents can contribute inherited knowledge back to DataHub, including:

  • Incident state and resolution
  • Agent-generated documentation
  • Tags and governance context
  • Repair summaries
  • Validation evidence
  • Responsible agent and execution timestamp
  • Auditable action notes

The next person or agent inspecting the asset can inherit the knowledge created during the repair.

10. Ownership, domains and stable identities

Each mission works with canonical DataHub URNs rather than informal table names.

Assets are connected with organizational context such as:

  • Data platform
  • Domain
  • Owner
  • Cancer cohort
  • Schema
  • Queries
  • Lineage
  • Health status
  • Governance metadata

The user interface displays this as “DataHub Identity,” allowing a researcher or ML engineer to see which model, owner, cohort and governed dataset support the active digital twin.

11. Analytics Agent integration

The application includes an Analytics Agent integration for natural-language exploration of catalogued cancer datasets and their relationships.

The core verification path does not depend on an unverified language-model answer: MCP evidence, exact URNs, schemas, lineage and proof receipts remain visible independently. This keeps the demonstration auditable even when generative narration is unavailable.

Why DataHub is essential to OncoTwin 3D

Without DataHub, an agent may know that a feature table failed but not understand:

  • Which schema is authoritative
  • Which datasets produced it
  • Which model consumes it
  • Who owns the asset
  • Whether an incident is already active
  • What downstream systems could be affected
  • Which query created the table
  • What knowledge should be preserved after repair

DataHub supplies this missing organizational context.

OncoTwin 3D turns that context into an interactive agentic safety system: agents observe a cancer-data failure, investigate its lineage, calculate ML impact, generate a grounded repair, pause for human approval, validate recovery and write the resulting knowledge back to the context graph.

Verified DataHub coverage

  • 12 catalogued BigQuery tables
  • 1,227 metadata events in a successful ingestion run
  • 208 catalogued query entities
  • 7 cancer-context missions
  • 6 live MCP reads per mission
  • 42 condition-specific MCP operations per complete verification
  • Unique DataHub URNs for every mission
  • SHA-256 evidence receipts
  • Human-approved writeback
  • Incident raise-and-resolve lifecycle
  • No personally identifiable patient data

The biological failure scenarios are synthetic or de-identified and research-only. The DataHub metadata discovery, schemas, lineage, query evidence, proof receipts, incident governance and approved writeback operate against the live self-hosted DataHub deployment.

Challenges we ran into

• Making "context" provable, not just claimed. It's easy to fake a lineage graph in a demo. We had to build a live Proof Galaxy that performs fresh MCP calls (search, get_entities, list_schema_fields, upstream/downstream lineage, get_dataset_queries) on every proof capture, and exports a SHA-256-receipted JSON so nobody has to trust the UI on faith. • Keeping the human-approval boundary airtight. With six agents automating a real BigQuery repair and a real DataHub incident resolution, it was critical that nothing mutated without an explicit, single-use, Secret-Manager-backed approval value — and that this boundary was visible in both the UI and the exported receipts (mutation_performed: false until approved). • Standing up DataHub itself at hackathon scale. Running DataHub Core (frontend, GMS, MySQL, Kafka, OpenSearch) on a Compute Engine VM meant tuning disk size, swap, and networking (VPC + IAP tunnel) so a public Cloud Run judge site could reach a private catalog securely. • Schema-safe repair generation. Generating BigQuery SQL that actually matched the cataloged schema and contract (rather than a plausible-looking guess) required grounding Repair Engineer's output directly in the live DataHub schema/contract responses, not just the table name. • Making seven failure modes feel distinct, not repetitive. Cohort drift, schema mutation, multi-omic discordance, protein provenance rifts, and spatial microenvironment escape all needed their own visual grammar and their own DataHub-native data product, lineage, and contract — so each mission had to be independently wired end-to-end rather than templated.

Accomplishments that we're proud of

• A governed writeback loop that is genuinely undeniable: real DataHub incident → real blast-radius lineage proof → real consumer block → human approval → real BigQuery repair → real incident resolution → real metadata writeback → real read-after-write verification. • 7/7 reliability missions each with 6/6 successful live DataHub MCP reads, verifiable via scripts/18_verify_all_datahub_conditions.sh. • A judge-proof receipt system (Proof Galaxy) that exports a timestamped, SHA-256-verifiable JSON showing exactly what was read and confirming zero writes occurred outside the approval boundary. • Turning DataHub's metadata graph into something visible and replayable — blast radius as animation, incidents as agent constraints, governed writeback as inherited memory — rather than another dashboard bolted on top of a catalog. • A DataHub Causal Observatory: a thirteen-node spatial lineage organism that binds live, condition-specific MCP proof to lineage edges across all seven cataloged products, with a clear on-screen boundary between live evidence, counterfactual simulation, and human policy enforcement.

What we learned

• Context is the actual safety mechanism. The hard part of "safe AI agents on regulated data" isn't a smarter model — it's giving the agent canonical schema, lineage, ownership, and governance state before it's allowed to act, and DataHub's context graph does most of that work if you treat it as an operational input rather than documentation. • Blast-radius reasoning changes the decision. Blocking progression_scores only makes sense once you can prove it's downstream of progression_features — table names alone don't get you there; real lineage does. • Honesty about simulation boundaries matters as much as the demo itself. Being explicit that biological telemetry, RL consequences, and protein folding are research simulations — while the DataHub reads, GCP jobs, incident lifecycle and writeback are live — made the system more credible, not less. • Human-approval gates are a design surface, not an afterthought. Deciding exactly what pauses for approval (and proving nothing else mutates) took as much engineering as the repair logic itself. • The same pattern generalizes well beyond oncology — the block → repair → resolve → writeback loop applies to any regulated feature product (fraud, finance, manufacturing, supply chain) where "trust but verify" needs to be literal. What's next for OncoTwin

What's next for OncoTwin 3D — DataHub Cancer Context Mission Control

• Extend the governed-writeback pattern from the flagship Biomarker Completeness Crisis to fully automated (approval-gated) repair across all seven missions, not just condition incident lifecycles. • Generalize the mission framework beyond oncology into a reusable "DataHub safety mission" template for other regulated domains (fraud detection, financial features, manufacturing telemetry). • Deepen the Analytics Agent integration so natural-language questions can trigger /improve-context writebacks with richer context-quality scoring across all seven data products, not just the flagship one. • Replace the tabular Q-learning safety controller with a learned policy trained on real historical incident/repair outcomes, while keeping the same DataHub-grounded state representation. • Expand the 3D Causal Observatory into a persistent, multi-tenant view so a platform team could monitor blast radius and open incidents across many pipelines simultaneously, not just a single judge-triggered mission.

# How Q Learning was used in the project

How OncoTwin Uses Q-Learning in Its Missions

OncoTwin uses a lightweight, tabular Q-learning safety controller to decide the next operational action during each cancer-data mission.

Q-learning does not recommend cancer treatment, predict patient outcomes or learn from identifiable patient records. It operates at the data and ML-governance level: deciding whether an ML pipeline should continue, pause, investigate, repair, validate or escalate for human approval.

DataHub provides context; Q-learning selects an action

The two components have distinct responsibilities:

  • DataHub supplies trusted context: canonical URNs, schemas, lineage, queries, ownership, health status and active incidents.
  • Q-learning evaluates the current safety state and selects the most appropriate governed action.
  • Agents execute the selected investigation or repair step.
  • Human approval remains mandatory before any mutation.
  • DataHub writeback records the outcome so future users and agents inherit the knowledge.

The policy therefore operates on DataHub-grounded observations rather than guessed metadata.

Mission state

For each step, the controller constructs an operational state similar to:

state = {
    data_trust_band,
    active_incident,
    schema_compatible,
    quality_failure,
    drift_band,
    downstream_impact,
    model_risk_band,
    model_blocked,
    repair_generated,
    human_approval,
    validation_passed
}

Examples of live DataHub evidence contributing to this state include:

  • Whether the selected dataset exists in DataHub
  • Whether the expected schema fields are present
  • Whether a DataHub incident is active
  • How many downstream assets are affected
  • Whether an ML scoring table appears downstream
  • Whether the relevant transformation query is available
  • Whether the repair has passed validation
  • Whether the incident has been resolved

Biological failure signals are synthetic or de-identified. The metadata used to ground the operational decision is retrieved live from DataHub.

Available actions

Depending on the mission state, the Q-learning controller can select an action such as:

  • inspect_context
  • inspect_schema
  • trace_lineage
  • block_model_consumption
  • generate_repair
  • request_human_approval
  • execute_approved_repair
  • validate_quality
  • resolve_incident
  • write_knowledge_to_datahub
  • unblock_model
  • escalate_for_review
  • continue_monitoring

The controller cannot bypass the approval gate. Selecting a repair action only prepares the mutation; a human must authorize its execution.

Q-value update

The policy learns the expected long-term value of taking action (a) in state (s):

[ Q(s_t,a_t) \leftarrow Q(s_t,a_t) + \alpha\left[ r_{t+1} + \gamma\max_a Q(s_{t+1},a) - Q(s_t,a_t) \right] ]

Where:

  • (s_t) is the current DataHub-grounded safety state.
  • (a_t) is the selected operational action.
  • (r_{t+1}) is the reward produced by the resulting transition.
  • (\alpha) is the learning rate.
  • (\gamma) controls the importance of long-term safe recovery.
  • (\max_a Q(s_{t+1},a)) represents the best expected action from the next state.

The objective is not to maximize biological progression or treatment response. It is to maximize safe, evidence-backed recovery of the data and ML system.

Reward design

The reward function encourages safe and auditable behavior:

[ R = w_sS + w_eE + w_vV + w_gG - w_uU - w_mM - w_dD ]

Where:

  • (S): unsafe model consumption was prevented.
  • (E): the action was supported by DataHub evidence.
  • (V): repair and quality validation succeeded.
  • (G): governance requirements and human approval were respected.
  • (U): an unsafe asset was consumed.
  • (M): an unauthorized mutation was attempted.
  • (D): unnecessary model downtime or repeated actions occurred.

Positive rewards are generated when the controller:

  • Grounds a decision in DataHub metadata
  • Blocks an unsafe downstream model
  • Requests approval before mutation
  • Produces a schema-compatible repair
  • Passes quality validation
  • Resolves the incident
  • Restores the ML pipeline safely
  • Contributes an audit note or documentation back to DataHub

Penalties are generated when the controller:

  • Allows an unhealthy dataset into model consumption
  • Acts without sufficient metadata evidence
  • Attempts an unapproved write
  • Generates schema-incompatible code
  • Unblocks a model before validation
  • Repeats unnecessary actions
  • Leaves an incident unresolved

Example: Biomarker Completeness Crisis

The strongest Q-learning storyline is the Biomarker Completeness Crisis mission.

Initial state

data trust: degraded
active incident: yes
signal columns: incomplete
downstream ML asset: progression_scores
model blocked: no
repair validated: no

Policy decision

DataHub lineage shows that progression_scores depends on the unhealthy progression_features table.

Allowing the model to continue would receive a large negative reward. Blocking consumption and investigating the schema has a higher expected long-term value.

The policy therefore selects:

block_model_consumption

Recovery sequence

The state changes as the agents complete their work:

Detect failure
      ↓
Read DataHub context
      ↓
Discover downstream ML impact
      ↓
Block model consumption
      ↓
Generate schema-grounded repair
      ↓
Request human approval
      ↓
Execute approved repair
      ↓
Validate quality
      ↓
Resolve DataHub incident
      ↓
Write inherited knowledge
      ↓
Unblock model

A large positive terminal reward is produced only when validation passes, the incident is resolved and the model is safely unblocked.

How the seven missions use the policy

Mission DataHub-grounded risk Preferred safe action
Biomarker Completeness Crisis Missing progression signals affecting downstream scores Block, repair, validate and resolve
Tumour-State Progression Surge Unexpected state-transition evidence Trace provenance and require research review
Cancer Cohort Drift Distribution drift affecting model reliability Gate consumption and initiate retraining assessment
Genomic Schema Mutation Breaking genomic schema change Block incompatible pipeline and generate a schema-aware patch
Multi-omic Biomarker Discordance Conflicting evidence across molecular sources Hold automated interpretation and escalate for evidence review
Protein Conformation Evidence Rift Inconsistent or weakly governed conformation evidence Quarantine affected evidence and inspect provenance
Tumour Microenvironment Escape Spatial-state shift affecting downstream interpretation Increase monitoring and gate affected model outputs

All seven missions use the same governance-oriented state-action framework, but each is grounded in a different DataHub asset, schema, lineage graph and query history.

Live mode versus verified replay

In Live mode, the application captures fresh DataHub MCP evidence, constructs the current mission state and evaluates the next policy action.

In Verified Replay mode, the application replays the previously recorded state, action, reward and agent evidence without repeating mutations. This lets judges inspect the complete decision trajectory safely.

Why combine Q-learning with DataHub?

Q-learning alone could optimize against incomplete or incorrectly assumed state. DataHub alone can describe the data estate but does not choose how an autonomous workflow should react.

OncoTwin combines them:

DataHub context
      +
Q-learning safety policy
      +
Specialized agents
      +
Human approval
      +
Governed writeback

This creates a closed-loop ML safety controller in which every action is grounded, explainable, approval-aware and auditable.

The result is an interactive digital twin of both the cancer-research state and the data platform supporting it. The visualization shows not only what changed, but why the policy selected an action, what evidence supported it, what reward was received and how the system returned to a trusted state.

Safety statement

The Q-learning controller governs data and ML operations only. It does not provide patient diagnosis, treatment selection, drug prescription or clinical advice. All biological scenarios are research-only simulations using synthetic or de-identified concepts.

Built With

  • agent-context-kit
  • analytics-agent
  • and
  • bigquery
  • cloud-run
  • compute-engine
  • datahub-core
  • datahub-mcp-server
  • datahub-skills
  • docker
  • fastapi
  • langchain
  • manager
  • secret
  • tabular
  • three.js
  • vertex-ai-gemini
Share this project:

Updates

Submission history