TraceGuard — Governed AI Quality Containment

A quality model failed silently. DataHub identified the governed decision path; TraceGuard resolved it into physical products and caught an already-shipped LOT that timestamp triage missed.

In five controlled runs, Agent + DataHub found the exact model-incident action set every time. The same Agent with the same factory tables, but without governed model identity, safely abstained in all five runs instead of guessing. Every result is public.

Inspiration

Production ML monitoring usually ends with a model alert or a data-lineage graph. In a factory, that is only the start. When an inspection model silently passes defects, the operational question is physical: which production LOTs, inventory units, and shipments were exposed, and what is the deterministic minimum-safe scope under declared rules?

TraceGuard turns DataHub context into that governed incident-response boundary.

Use of DataHub

TraceGuard investigates two synthetic battery-cell inspection incidents across a digital-to-physical path:

camera → observations → features → model version → decisions → LOTs → inventory → shipments

The Agent uses DataHub catalog search, lineage, entity context, and a bounded read-only MLModel deployment-context tool to identify the authoritative incident selector. TraceGuard then joins that selector through operational DuckDB rows to physical LOTs and shipments.

Sensor lineage excludes same-time LOTs routed through another camera. Governed MLModel metadata maps a production model version to an opaque inference route. TraceGuard uses that governed selector to find LOT-002, whose delayed, backfilled decision falls outside the incident time window.

After approval, TraceGuard writes the governed result back as an Incident, structured properties, tags, terms, descriptions, a decision document, model deprecation, and Incident status. Five writes use DataHub MCP; three narrow GraphQL wrappers cover self-hosted gaps. Completion is reported only after the new decision document is read back successfully.

Technical Execution

The LLM investigates and explains. It never assigns risk tiers and cannot execute factory actions. A deterministic Python engine applies product susceptibility, downstream QMS evidence, inventory state, and shipment status:

• RED: recall released product;
• AMBER: hold and create reinspection while the LOT remains controllable;
• GREEN: clear the LOT with explicit product or QMS evidence.

In the hero model-version incident, 15 exposed LOTs / 77 cells become 7 action LOTs / 42 cells after 8 LOTs / 35 cells are cleared by evidence. Two LOTs require recall; five require hold and reinspection. The affected set includes delivered LOT-002, which timestamp triage misses.

The camera fixture is a positive control: 12 exposed LOTs / 47 cells become 7 action LOTs / 28 cells, and both Agent profiles are exact 5/5. No DataHub accuracy advantage is claimed there.

A quality lead approves the exact SHA-256 plan fingerprint. Any plan change invalidates approval. The FastAPI Mock MES then performs only allow-listed, idempotent recall, hold, reinspection, and rollback operations, with receipts bound to the approved plan.

The stack includes DataHub Core 1.6, DataHub MCP Server 0.6 with mutations explicitly enabled, OpenAI Responses API, DuckDB, a real logistic-regression pipeline, FastAPI, Streamlit, GraphQL, Docker, pytest, Ruff, and GitHub Actions. The suite passes 59 tests on Windows; a fresh public Debian clone separately passed 55 tests, Ruff, fixture regeneration, and bash checks.

Originality

TraceGuard does not rebuild a catalog feature. Its new layer is a Digital-to-Physical Exposure Graph plus a Minimum Containment Engine: governed lineage identifies the decision path, while deterministic evidence resolves the minimum-safe physical scope under declared rules.

The public fixture contains exoneration evidence and temporal decoys. The controlled evaluation holds the model, reasoning effort, prompt, Agent code, DuckDB data, and permissions constant. Only DataHub context is removed. On the model incident:

Condition	Runs	Recall	Precision	FP	FN
Agent + DataHub	exact 5/5	100%	100%	0	0
Same Agent, tables only	safe abstention 5/5	0%	0%	0	7
Timestamp heuristic	1 run	85.7%	50%	6	1

The tables-only Agent returns UNRESOLVED_NO_GOVERNED_MAPPING rather than inventing model identity from route statistics. Named adversarial frequency, recency, LOT-ID, unit-count, and prediction heuristics are also published—including the heuristics that guess correctly. A publication gate changes claim wording, never result rows.

Real-World Usefulness

The manufacturing demo makes the consequence visible: model decisions can reach held inventory and already-shipped products. The same decision-to-consequence pattern applies to loan applications scored by a faulty model, insurance claims processed under a broken version, and content-moderation decisions made under a faulty policy.

TraceGuard sits above—not in place of—data platforms, MES, QMS, and model monitors. The synthetic prototype demonstrates a vendor-neutral integration boundary without claiming commercial factory connectors.

Submission Quality

The Apache-2.0 repository includes one-command macOS/Linux setup, a Windows path, an English README, architecture documentation, a reusable Manufacturing Quality Containment Skill, complete Agent traces, governed write-back receipts, browser E2E evidence, and an examples/ index for review without running DataHub Core.

The project also produced two actionable upstream contributions: an open, unmerged upstream PR for a Windows CP949 quickstart UTF-8 fix with regression coverage and a reproduced self-hosted mlModelDeployment GraphQL issue. Unsupported Agent Registry and Change Proposal claims were removed after live Core capability gates.

All factory data, model artifacts, incidents, and actions are synthetic. AI coding assistants, including Codex, were used for implementation and review. Every submitted behavior is backed by a test, public receipt, or explicit limitation.

Repository and test instructions: https://github.com/JJI-Hoon/traceguard#quick-start

Evidence index: https://github.com/JJI-Hoon/traceguard/tree/main/examples

Built With

• datahub-core
• datahub-mcp-server
• datahub-skills
• docker
• duckdb
• fastapi
• github-actions
• graphql
• openai-responses-api
• pytest
• python
• ruff
• scikit-learn
• streamlit