Token Guard

A governance-native dedup gate for AI agents, built on DataHub. Refuses to serve stale or MNPI-restricted results - checks freshness and governance before ever hitting cache.

Comment

Inspiration

2026 is the year the token bill came due. The FinOps Foundation's conversation has flipped from "ship agents fast" to "we need guardrails," and the Linux Foundation stood up a Tokenomics Foundation specifically to standardize token-cost discipline. A research paper we read while scoping this build — Salim et al., "Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering" (MSR '26) — found that in one agentic-coding benchmark, code-review/refinement loops alone consumed ~59% of all tokens, largely because agents keep re-doing work a sibling agent (or their own earlier self) already did.

Almost all of the tooling forming up around this problem is observability — dashboards that tell you where the tokens went, after they're gone. Almost nothing prevents the redundant spend at execution time, because doing that safely requires knowing not just that two agent actions look alike, but that they are safe to treat as the same: same lineage, same business meaning, same governance boundary. That's exactly the context a metadata graph like DataHub carries and a bare lineage feed doesn't. We picked the highest-margin case of this problem — financial- research agents re-reading and re-analyzing the same governed filings, earnings calls, and news — because it's also the case where serving a wrong cached answer is a compliance incident, not a rounding error, which forces the governance story to be real instead of decorative.

What it does

TokenGuard is a proxy that sits in front of enterprise-agent tool calls, built as a small team of governance agents rather than a passive cache:

  1. Intent resolution grounds a raw tool call in DataHub — reading search, get_entities, get_lineage, get_lineage_paths_between — to produce an Intent: the entities touched, a bounded lineage subgraph, a normalized parameter vector, and a governance fingerprint (glossary terms, domain, classification/PII tags).
  2. Matching checks a new intent against prior ones, cheapest first: an exact-hash cache (Tier 1), then Weisfeiler-Leman structural similarity, cosine similarity over parameters, and a governance-equivalence check that refuses to dedup two structurally-identical pipelines that mean different things (a revenue pipeline vs. a cost pipeline over near-identical tables, distinguished only by glossary term).
  3. The governance gate is the safety backstop before anything is served: schema drift, lineage change, a freshness check, and a governance-boundary check against the requester's scope vs. the entity's live domain/classification/ownership. Bias throughout: when in doubt, re-execute. Every decision carries a stable TG-GATE-*/TG-DEDUP-* reason code.
  4. Closed-loop write-back: every decision writes a tokenguard.dedup_record structured property back onto the touched DataHub entities — so the fact is visible on the graph, not only living in TokenGuard's process memory, durable across restarts and separate Proxy instances.
  5. A circuit breaker quarantines a runaway agent (stale schema, broken retry, reasoning loop) and writes a tokenguard.breaker_trip marker back to the graph so the event is visible to an auditor, not buried in a log file.

The financial-research flagship demonstrates three concrete scenarios: a freshness gate that forces re-execution the moment a 10-Q is restated even though the schema is unchanged; an MNPI-compliance gate that refuses a cached serve to an agent that self-asserts a clearance it doesn't actually have (the F1 scope-escalation attack, rejected before the gate or matcher ever runs); and a saturation gate that stops paying for a scraping agent's calls the moment additional sources return only duplicated information.

How we built it

DataHub sits behind an abstract client interface whose method surface mirrors what TokenGuard needs — search, get_entities, get_lineage, get_lineage_paths_between, list_schema_fields, get_dataset_queries, upsert_structured_property, get_structured_property, add_tag. The real, DataHub Core–backed implementation uses the acryl-datahub SDK's DataHubGraph directly against GMS (not the MCP Server) — a self-hosted DataHub Core instance, no DataHub Cloud, no license gate. Its define → write → read-back structured-property loop is live-confirmed end-to-end against a real DataHub Core v1.5.0.6 instance.

An in-memory implementation of the same interface backs every unit/e2e test and the demo, so the whole suite and examples/demo.py run hermetically — no live DataHub, no Redis, no network, no Docker required to build or test. docker-compose.yml separately provisions a real DataHub Core stack for the live path; it's additive, never required.

The public judge-testable URL (https://tokenguard.itinerario.io) runs this live path continuously: a fixed, curated workload on a loop against a real, self-hosted DataHub Core instance — not organic production traffic — but every decision and write-back it renders is a real Proxy.handle() outcome, not scripted.

Four points in the pipeline are load-bearing on DataHub's unified graph specifically — lineage plus glossary plus domain plus classification plus real query history plus a mutation API in one place — and would break or silently misbehave with a bare lineage feed swapped in: the governance fingerprint, the governance-equivalence check (which fails closed when both sides are untagged — absence of tagging is treated as absence of evidence, not evidence of a match), the gate's governance-boundary check (which re-reads live classification at serve time, not a cached baseline), and the closed-loop structured-property write-back.

Challenges we ran into

The DataHub Python SDK's structured-properties path has real, silent footguns. The official tutorial has no Python tab at all — only CLI/GraphQL/OpenAPI — and the two natural SDK affordances are easy to misuse in ways that fail silently rather than loudly. StructuredProperties.create looks like an instance builder but is actually a @staticmethod whose first argument is a YAML file path, not a graph — calling it the "obvious" way silently passes the graph in as the file argument and fails. And there's no single-property value read: the tutorial's "Read a single Structured Property" section reads a property's definition, not its value on a given entity — the value only comes back as part of the entity's whole structuredProperties aspect, which you fetch and filter yourself. Separately, wiring the write-back loop against the live instance also surfaced that GlossaryTermsClass requires an auditStamp argument that isn't obvious from the type alone. We turned the SDK-documentation gaps into a real open-source contribution back to DataHub itself — a documentation PR adding a Python tab to the Structured Properties tutorial's Create and Read sections, verified twice: once against DataHub's own source, once against our live GMS run. Open for review at datahub-project/datahub#18619.

Choosing where to spend the "algorithm" budget — and where not to. It would have been tempting to chase a fancier matcher: LSH/MinHash candidate blocking to make near-duplicate retrieval sub-linear, Merkle subtree hashing to catch more structural clones, per-item adaptive similarity thresholds tuned by governance category. We designed all three and then deliberately did not build them, for reasons that came from measuring first rather than assuming:

  • The scan they'd optimize isn't the bottleneck. Tier-2 matching without a blocking layer is a linear scan over the candidate pool — O(N) per incoming call, each comparison cheap and in-memory. At the traffic volumes we could actually measure, N stayed in single digits, so O(N) is already sub-millisecond. The dominant real cost per decision is a handful of serial DataHub round-trips (resolve, gate, per-candidate refetch, write-back) — network I/O that dwarfs any in-memory complexity difference between a linear scan and a sub-linear index at this scale. Building a fancier index would have optimized a term that isn't on the critical path.
  • The "obvious" optimization has a correctness cost, not just an engineering cost. Merkle-based structural hashing was proposed to sit on the Tier-1 exact-match path — the one tier that bypasses every governance check — with no allowlist distinguishing a semantically-irrelevant parameter change from one that should force a re-check. Per-item adaptive thresholds ("looser for Public data") would reduce precision on the one tier that's the sole discriminator within a governance class, with no tested precision floor. In a system whose entire premise is "when in doubt, re-execute," loosening either to chase a marginally higher hit-rate is exactly the tradeoff the gate exists to prevent.
  • Even the proposed retrieval key was measurably wrong for the workload it targeted. The planned LSH blocking key (a MinHash over touched-URN sets) is misaligned with what structural matching actually compares (a URN-agnostic shape) — it would silently drop true structural duplicates that don't share table names, and it degenerates to one giant, unfiltered bucket under exactly the high-redundancy traffic pattern (many agents hammering the same table) it was meant to help with. An index that returns zero shrinkage precisely where redundancy is highest isn't a premature optimization — it's the wrong one.

So the complexity story here isn't "we ran out of time" — a real measurement pass showed the return on investing engineering time in matcher sophistication was near-zero and carried a real safety cost, while the freshness/governance-boundary gate had neither problem. We went deep on the gate specifically because the numbers said that's where the ceiling actually was, not the matcher.

Being honest about what "governance" actually buys you took real iteration. It would have been easy to ship a headline dedup-hit-rate number and call it a day. Instead, measuring the reference implementation against its own adversarial catalog surfaced an uncomfortable truth: on a toy-scale catalog, structural/parametric matching (Tier 2) contributes essentially zero additional hits over a correctly-scoped exact-match cache (Tier 1) — the real value isn't in catching more duplicates, it's in refusing to serve the duplicates that shouldn't be served (a restated filing, an MNPI boundary). Reporting that honestly, instead of picking a friendlier benchmark, changed how we framed the entire project: the moat is the gate, not the matcher.

Reason codes and audit trails needed to be real, not decorative. Early iterations logged decisions to process memory only — useful for a demo, useless to a compliance reviewer. Every decision now lands in a shared, cross-Proxy AuditLedger keyed by a stable reason code, exportable as a Markdown report or raw JSONL, specifically so the artifact answers "prove it" instead of just "trust me."

What we learned

  • A dedup engine's hardest problem isn't finding duplicates — it's knowing when a duplicate has stopped being safe to serve. Freshness and governance-boundary checks did more real work in this project than any similarity algorithm.
  • DataHub's real value here is as a unified source of truth, not any single API surface. Lineage alone, or classification alone, doesn't get you the governance-equivalence check or the live-boundary re-check — it's specifically the combination, in one graph, that makes the gate possible.
  • Measuring honestly is harder than it sounds, and worth it anyway. It's tempting to report the number that makes a project look best. The more useful number was the one that told us where the real value lived — and that discipline is now baked into docs/MEASUREMENT_REPORT.md and the project's honest-limitations section rather than left as a one-time exercise.
  • Contributing upstream is cheap once you've already done the hard part. The DataHub documentation gap we hit cost us real debugging time; writing it up as a PR for the next person cost very little more, once we understood the actual failure mode.

What's next

  • Real identity binding (mTLS/JWT/IdP) behind the existing ScopeResolver seam, so the F1 self-escalation fix becomes cryptographic rather than transport-trust.
  • A genuinely async write-back trigger (background worker / batch boundary) so "off the critical path" is a production guarantee, not something only the hermetic test harness calls explicitly.
  • Measuring Tier-2 and saturation at real traffic volume — the toy-scale near-zero result is expected at this scale, not a verdict on the mechanism, and a real design partner's traffic is the only way to actually test it.
  • Circuit-breaker recovery — a half-open/throttled probe state instead of permanent quarantine.
  • The coding-dedup transfer. Research-dedup and coding-dedup are the same shape — N agents independently re-ingesting one shared, governed source. The same resolver/matcher/gate/write-back that dedups financial research transfers directly to the SDLC code-review loop, which independent measurement puts at ~59% of agentic-coding token spend. We led with finance for this submission (cleanest governance story, safety-critical freshness) as evidence the engine is general, not as a second demo to build in one hackathon window.

Built With

  • acryl-datahub
  • apache-2.0
  • azure
  • blake3
  • cloudflare-tunnel
  • cosine-similarity
  • datahub
  • docker
  • docker-compose
  • github-actions
  • httpx
  • numpy
  • pytest
  • python
  • redis
  • ruff
  • starlette
  • streamlit
  • uvicorn
  • weisfeiler-leman-algorithm
Share this project:

Updates