Praxis - evidence-bound remediation agent.

Praxis takes that one narrow, expensive incident all the way to a mergeable fix, and refuses to hand you anything it cannot prove.

Comment

Praxis — the evidence-bound remediation agent for DataHub-governed data products

Prove it before you propose it.

The gap

Ask any catalog agent about a schema change and it will give you a good paragraph. Ask it to fix the break and you get generated code you cannot trust, because nothing connects the code to the metadata that justified it.

So the expensive incident stays manual. A source column is renamed. Nothing breaks loudly. A dbt model keeps projecting the old field, a production ML model keeps scoring on stale features, and a business-critical dashboard keeps looking fine — until someone notices, three weeks later, by hand.

What Praxis does

Praxis takes that one narrow, expensive incident all the way to a mergeable fix, and refuses to hand you anything it cannot prove.

It reads the exact DataHub schema and lineage for a specific dataset URN, inspects the real dbt files in your repository, generates the smallest safe compatibility patch, verifies it in isolation, stops for a named human decision, re-verifies after that decision, and then publishes only the outputs an external system actually confirmed.

The differentiator is not another incident summary. It is a proof-carrying change set. Every candidate ships with DataHub evidence URNs, before/after SHA-256 hashes of every file, the full verification result including the checks that did not run, and a rollback instruction.

The run, end to end

1 · Exact coordinates, not a search. Praxis starts from a dataset URN, a missing field, and an observed replacement. DataHub returns the observed schemaMetadata, ownership, the description, the failed assertion, and — this is the part that matters — a column-level lineage path proving the old field is still projected downstream. Entity lineage is never accepted as column proof.

2 · Blast radius. Entity lineage establishes the wider impact: a dbt staging model, a Feast feature table, an MLflow model, its SageMaker deployment, and a critical Looker queue. The risk score is composed only from evidence terms — failed assertion, confirmed column path, production entities, criticality, usage, missing ownership. Every term traces back to a URN.

3 · Generation against real files. Praxis reads your actual dbt model and contract, hashes both, and finds exactly one safe projection to repair. The patch is one line — alias the new column back to the contracted name, with the cast type read from the contract's declared data_type — plus switching contract: enforced: true on so dbt rejects the next drift itself. It refuses on ambiguous SQL, a missing contract field, an absent data type, an oversized file, or any path outside the configured repository.

4 · Fail-closed verification. Seven required checks: the rename is proved by observed schema; a column path is confirmed; the patch is minimal and preserves every projected output; there is no destructive SQL; the enforced contract matches every projected column exactly; the provenance manifest is internally consistent; the repository inputs still hash the same. Validation runs against a temporary copy. When dbt is installed, a real dbt parse runs as an eighth required check and the tier becomes dbt_verified; when it is absent the check is recorded as skipped, never as passed. If a required check fails, there is no approval button.

5 · The human gate. Approval requires a named identity and is bound to the exact candidate hashes. On approval Praxis re-reads the repository, repeats verification, and compares the hashes again — so a review that went stale while the human was deciding can never be applied. The source repository is never overwritten.

6 · Confirmed outputs only. With GitHub configured, one Git tree and one commit become one draft pull request. With DataHub configured, praxis.verified-impact is applied only to URNs DataHub acknowledged in its mutation response, and a hash-bound memory note is appended to the source dataset so the next engineer — or the next agent — inherits what this run proved. An attempted write is never reported as a successful graph delta; a connector failure produces a partial result naming the exact confirmed subset.

7 · Tamper-evident by construction. Every handoff lands in a SHA-256 chained ledger in SQLite: event_hash = SHA256(canonical_event ‖ sequence ‖ prev_hash), with a unique (run_id, sequence) index to prevent reordering. The UI verifies the chain on every snapshot. Startup recovers queued, running, applying, and approved-waiting runs; apply is safe to resume because branches and memory notes use stable run identifiers.

Try it in 60 seconds — no credentials

python -m praxis        # Python 3.10+, zero dependencies

Open http://127.0.0.1:8787, select Resolve evidence & verify, inspect the Evidence and Candidate tabs, and approve or reject.

The backend loads a packaged DataHub-shaped incident fixture, reads the intentionally broken dbt project under sandbox/dbt_project, generates new files, validates them, persists a chained ledger, and serves the resulting artifacts. Results are computed, not replayed.

For the same real dbt parse gate used in the Docker image:

pip install -r requirements-dbt.txt && python -m praxis

Or fully self-contained, including the pinned DataHub SDK:

docker compose up --build

How Praxis uses DataHub

Not as a lookup table — as both the source and the destination of governed context.

Purpose DataHub surface
Exact observed schema, ownership, description GraphQL dataset { schemaMetadata, ownership, editableProperties }
Wider entity blast radius GraphQL scrollAcrossLineage with degree filters
Column-level field paths (the proof) Official Python SDK client.lineage.get_lineage(source_column=…)
ML deployment edges invisible to lineage search DataHubGraph.get_aspect(urn, MLModelPropertiesClass)
Publishing verified impact GraphQL addTag mutation, confirmed per URN
Publishing organizational memory GraphQL updateDescription, appending without clobbering
Discovery and lineage inside an agent loop DataHub MCP Server, composed with ours in .mcp.json
Agent-facing catalog workflow DataHub Skill in contrib/skills/, upstream format
Reproducible live test graph datahub docker quickstart --version v1.6.0 + our own seed script

One command to the live path

The sandbox is a fixture and says so. The DataHub-backed path is one command:

python setup/live_quickstart.py

That installs the pinned SDK, brings up DataHub Core 1.6.0, waits for GMS to answer GraphQL, and seeds a self-contained nine-entity graph — post-rename source schema, two-hop column lineage, three ML features, a production model, its model group, its SageMaker deployment and a critical dashboard. No warehouse, no datapack, no credentials. --serve starts Praxis on it; --stop tears it down.

After that, every schema field, owner, lineage edge and column path Praxis shows came out of GMS over the wire.

Sample outputs

Four inspectable artifacts per run, checked in at examples/verified-schema-drift/ with a walkthrough in examples/README.md:

  1. the one-line dbt compatibility repair;
  2. the dbt contract with enforcement enabled;
  3. a provenance manifest with DataHub URNs and input/output hashes;
  4. a complete verification report, including skipped checks.

All four are generated by the same code path a judge runs locally.

What is built

Verified end to end. DataHub Core 1.6.0 live schema, ownership, two-hop column lineage, and a nine-entity dataset-to-dashboard/model/deployment blast radius. A full live approval created nine confirmed DataHub tags and one hash-bound memory note with zero write errors. A Dockerized run completed the combined transaction: two confirmed column paths, nine impact entities, a real dbt parse, nine confirmed tags, one memory note, and an atomic draft pull request with exactly one commit and four expected files — with the base branch SHA and the local checkout unchanged. A contaminated cross-platform dbt partial-parse cache correctly failed closed before approval.

Limitations. Praxis supports a conservative dbt schema-rename repair, not arbitrary incident classes. SQL generation requires exactly one bare projection and an existing contract field with a declared data_type. The packaged healthcare graph is a test fixture derived from the contest scenario, not a captured live response. Live permissions depend on your deployment; the validated target is DataHub Core 1.6.0. Praxis creates a draft PR — it never merges, never deploys, and never changes production traffic. Production risk remains until your team merges. The product UI says this on its own front page.

Where the agent is, and where the proof is

Praxis is two layers, and the split is the design.

The agent layer is where language models belong. Praxis ships as an MCP server (python -m praxis.mcp, five tools) and as a DataHub Skill, so a real agent — Claude Code, Cursor, Codex, an SDK agent — does the open-ended work it is good at: finding the dataset, reading the graph, judging whether an incident is worth acting on, explaining the change to a human. contrib/skills/ pairs the DataHub MCP Server for discovery with the Praxis MCP server for remediation, and .mcp.json in the repo wires both up.

The remediation layer is deterministic, and that is what makes it trustworthy. Once the agent has named an exact incident, generation and verification are evidence-constrained code, not inference. Every proposed token traces to source metadata or repository content; there is no prompt-injection path from catalog text into generated production SQL; and the judge path reproduces with zero secrets.

That boundary is enforced, not just described. The MCP tools let an agent prepare, inspect and explain — and refuse to let it approve. praxis_approve_remediation requires a named human identity plus an explicit human_approved flag the tool description forbids an agent from setting on a person's behalf, and a run with a refused approval stays in waiting with nothing applied. Tests assert exactly that.

The trade: this narrows the incident classes Praxis handles to ones a deterministic generator can prove. We would rather ship one repair that carries proof than ten that carry prose.

Built and tested

Python 3.10+ with zero runtime dependencies for the sandbox path — stdlib http.server, sqlite3, urllib, hashlib. Vanilla HTML/CSS/JS frontend, no build step. Optional pinned integrations: acryl-datahub==1.6.0.15, dbt Core 1.12.0 + dbt-duckdb 1.10.1. Non-root Docker image. Apache 2.0.

Automated tests cover the DataHub 1.6 GraphQL query shapes, live evidence merging column proof with entity impact, two approval races — at worker handoff, and an approval landing while the preparation worker still holds the run's slot — restart-safe state, rejection safety, candidate generation, path traversal, SQL injection in field inputs, API authorization and the approval token, SQLite old-schema migration, connector truthfulness (a disabled GitHub raises instead of fabricating a PR), description-append preservation, and hash-chain tamper detection. CI runs the suite on Python 3.10 and 3.12, plus a judge-path job that installs dbt, runs dbt parse on the sandbox project, builds a wheel, and builds the Docker image.

What we would build next

Broaden the incident taxonomy beyond schema rename — type widening, nullability changes, and partition-key drift are the same transaction shape with a different generator. Then a batch mode that proves and bundles a whole day of drift into one reviewable PR.

Built With

Share this project:

Updates