RevokeGraph

When data consent changes, RevokeGraph finds every affected model, orchestrates an approved replacement, and proves the withdrawn data no longer reaches production.

Comment

Inspiration

Data consent is treated as static, but real permission changes after models have already been trained.

Existing governance systems can prevent future access to a dataset, yet they rarely answer the harder operational question: which production models were influenced by that data, what must change, who must approve it, and how do we prove the change reached production?

We built RevokeGraph to close that gap. It treats a withdrawal as an executable model-governance event rather than another deletion ticket.

What it does

RevokeGraph is a DataHub-native model revocation control plane.

When a contributor withdraws training permission, RevokeGraph:

  1. Reads the governing covenant, ownership, and model lineage through the official DataHub MCP server.
  2. Identifies every affected feature, model component, model version, owner, and deployment.
  3. Compiles a content-addressed Revocability Manifest describing the exact required consequence.
  4. Pauses at a LangGraph checkpoint for an authorized human reviewer.
  5. Retires or retrains only the affected source-isolated component.
  6. Builds a new immutable model revision.
  7. Blocks release unless five utility, exclusion, and community-impact gates pass.
  8. Requires a separate approval before writing the consequence back to DataHub.
  9. Verifies that the withdrawn source has no lineage path to the new production revision while retained sources still do.
  10. Produces a downloadable Model Revocation Receipt connecting authority, model consequence, and serving proof.

RevokeGraph does not claim to erase influence from an old model artifact. It verifies a precise and defensible outcome: the replacement model excludes the withdrawn cohort and the superseded artifact is no longer serving.

How we built it

DataHub is the system of context and evidence. Its official MCP server supplies governed metadata, ownership, downstream lineage, entity details, and exact path tracing. Approval-gated DataHub SDK operations register the replacement model, immutable deployment revision, supersession state, and revocation receipt.

A deterministic policy compiler converts covenant metadata into a Revocability Manifest. The manifest maps governed cohorts to source-isolated model components, permitted revocation actions, responsible reviewers, service-level objectives, and validation thresholds.

LangGraph coordinates the resumable human-in-the-loop workflow. It pauses at three consequential boundaries:

  • replacement approval;
  • metadata write-back approval;
  • postcondition verification authorization.

Approvals are bound to reviewer identity and exact content hashes. An approval for one plan, receipt, or write-back operation cannot authorize another.

The model layer uses scikit-learn and deterministic synthetic cohorts. RevokeGraph retires the withdrawn component, rebuilds the aggregate model, and evaluates ROC-AUC, recall, maximum community recall change, dataset exclusion, and prior-model supersession.

The reviewer workbench is built with Streamlit. Pydantic defines the domain contracts, SHA-256 content addressing protects evidence integrity, and Docker and Render provide the hosted deployment. GitHub Actions runs Ruff, 30 automated tests, a canonical dry run, and committed-evidence validation before deployment.

Challenges we ran into

The hardest challenge was proving a negative. Showing a new model in a catalog is straightforward; proving that a withdrawn source no longer reaches the new production revision requires both a zero-path check and a retained-lineage witness. Without the retained witness, an empty result could simply mean the entire graph was disconnected.

We also had to keep approval state stable across Streamlit reruns. LangGraph checkpoints, content-bound decisions, idempotent operations, and an explicit audit ledger made the workflow resumable without accidentally repeating side effects.

A third challenge was separating live execution from public demonstration. The hosted environment cannot safely expose DataHub credentials, so RevokeGraph clearly distinguishes live MCP execution from deterministic public replay of evidence captured through the official MCP server.

Finally, we resisted overstating the result. Replacement and exclusion are meaningful operational guarantees, but they are not the same as certified machine unlearning or mathematical erasure from a superseded artifact.

Accomplishments that we're proud of

  • Built a complete withdrawal-to-production control loop instead of a lineage visualization.
  • Made DataHub execution context part of planning, blast-radius analysis, write-back, and verification.
  • Implemented real LangGraph human-in-the-loop interrupts with approve, reject, stop, retry, and reset controls.
  • Bound every consequential decision to identity and content-addressed evidence.
  • Added five blocking model and community-impact gates.
  • Produced both negative lineage proof and a retained-source witness.
  • Created a portable Model Revocation Receipt with artifact, plan, operations, and evidence hashes.
  • Delivered a public, resettable reviewer workbench with downloadable audit evidence.
  • Shipped a containerized deployment with security headers, health checks, CI, browser smoke tests, and 30 passing tests.

What we learned

Metadata becomes much more valuable when it can drive an operational control loop.

Lineage tells us where influence may have propagated. Versioned governance metadata tells us why action is required. Human approval establishes authority. Model validation determines whether a replacement is safe. Postcondition verification proves whether the intended production consequence actually occurred.

We also learned that responsible AI infrastructure needs precise proof boundaries. A narrower claim supported by reproducible evidence is stronger than an ambitious claim the system cannot verify.

What's next for RevokeGraph

The next step is connecting the control plane to production model registries and deployment platforms such as MLflow, SageMaker, and Vertex AI.

We also plan to add:

  • durable PostgreSQL-backed LangGraph checkpoints;
  • OIDC authentication and tenant-scoped RBAC;
  • KMS-backed receipt signatures and append-only audit storage;
  • background execution, retries, leases, and revocation-SLO monitoring;
  • general withdrawal intake and multi-covenant policy administration;
  • partial-withdrawal retraining across additional model architectures;
  • strategy-specific verification for approximate and certified machine unlearning.

Our goal is to make revocability a standard property of production ML systems: if authority changes, the model consequence should be bounded, reviewable, and provable.

Built With

  • datahub
  • datahub-mcp
  • datahub-sdk
  • docker
  • github-actions
  • human-in-the-loop
  • langgraph
  • machine-unlearning
  • mlops
  • model-governance
  • playwright
  • pydantic
  • pytest
  • python
  • render
  • responsible-ai
  • ruff
  • scikit-learn
  • streamlit
  • uv
Share this project:

Updates