GlassBox

DataHub tells you what changed. GlassBox proves which AI decisions can no longer be trusted—and safely closes the loop.

Comment

Inspiration

DataHub gives organizations a trusted map of their data: where it came from, how it changed, who owns it, and what depends on it. But a consequential AI-agent decision can still become an orphaned fact. After the agent acts, teams are often left with logs that show that something happened, but not a durable, governed answer to:

  • Which exact data and tools shaped this decision?
  • Can we verify that evidence without retaining prompts, outputs, or business values?
  • If an upstream schema or policy changes tomorrow, which past decisions must be reviewed?
  • How can we recover safely without rewriting history?

GlassBox started from a simple idea: every consequential agent output deserves a verifiable bill of materials, and every material change to its governed context should produce an exact, actionable blast radius.

What it does

GlassBox is a DataHub-native decision-forensics and invalidation layer for AI agents.

An instrumented agent emits framework-neutral runtime evidence through OpenTelemetry. GlassBox compiles that evidence into a canonical, content-addressed Decision Bill of Materials (DBOM). The receipt commits to the agent run, dependency URNs, tool schemas, action inputs, and output digest; it can be signed with Ed25519 and independently verified.

GlassBox then publishes a raw-free receipt projection into DataHub and verifies it by direct readback. The governed asset remains the system of context; the receipt becomes the durable record of the decision that used it.

When DataHub emits a supported metadata change, the GlassBox Action:

  1. normalizes the change into a closed, versioned model;
  2. finds receipts that depended on the changed asset or field;
  3. evaluates materiality with deterministic rules and explicit reason codes;
  4. opens a content-addressed campaign;
  5. writes verified findings back to DataHub as incidents;
  6. quarantines affected receipts when policy requires it.

Operators can investigate the same evidence through the GlassBox console, a read-only MCP server, or the portable DataHub Agent Forensics Skill. Recovery is a separate, authorized workflow: a digest-bound approval permits an isolated replay, which publishes a successor receipt and an immutable supersession relation before the incident is closed.

The result is one connected lifecycle:

agent run → signed receipt → DataHub projection → metadata change → deterministic blast radius → investigation → authorized recovery

Why DataHub

GlassBox is useful because DataHub already owns the hard organizational context: canonical dataset identities, schemas, lineage, ownership, domains, policies, and change events.

We do not copy that governance layer. We bind agent decisions to it.

DataHub answers, “What changed in the governed data estate?” GlassBox adds, “Which AI decisions relied on it, why are they affected, and what evidence proves their current state?” Findings are written back into DataHub so the result becomes part of the metadata graph rather than knowledge trapped in another dashboard.

The integration is designed as reusable ecosystem infrastructure:

  • an installable DataHub Action for change-driven invalidation;
  • a read-only MCP server for decision forensics;
  • a portable DataHub Skill with deterministic helper scripts and evaluations;
  • typed receipt Documents and incidents that work with stable DataHub Core;
  • a native metadata RFC path for deeper upstream ownership;
  • an upstream pgQueue integration test prepared as DataHub Core PR #19004.

How we built it

GlassBox is an Apache-2.0 Python project with typed packages for runtime capture, DBOM construction, DataHub publication, policy evaluation, invalidation, forensics, and recovery.

The runtime layer supports direct SDK instrumentation, OpenTelemetry GenAI spans, LangChain/LangGraph, Google ADK, and MCP middleware. Strict compilation rejects ambiguous agent spans, dropped attributes, duplicate identities, and unsupported semantic schemas instead of guessing.

Receipt integrity uses RFC 8785 canonical JSON, SHA-256 content addressing, Merkle commitments, and Ed25519 signatures. Signer trust is operator-controlled: a signature proves key possession, while a separate trust policy decides whether that key is authorized.

PostgreSQL provides the shared operational authority for receipts, durable publication obligations, campaigns, leases, findings, and recovery checkpoints. DataHub remains the governed metadata authority. Every important write path uses direct readback verification, and retries reuse content-addressed identities rather than creating duplicate history.

Recovery runs inside an exact OCI image identity with networking disabled, a read-only root filesystem, dropped capabilities, resource ceilings, and a content-addressed isolation attestation. Supersession is append-only: the source receipt is never edited or deleted.

The hosted release candidate uses a multi-page operator console backed by real control-plane APIs. The production demonstration sends a genuine OTLP trace, publishes and rereads a signed receipt, applies a controlled schema change to a live DataHub dataset, computes the resulting blast radius, and drives the recovery workflow from persisted state.

Privacy and trust boundaries

GlassBox is deliberately raw-free across its governance boundary.

It stores and exposes governed URNs, digests, schema commitments, reason codes, verification results, and workflow state. It does not return prompts, model outputs, tool arguments, field values, credentials, or signing keys through DataHub, MCP, the console, or evidence reports.

This makes the evidence useful for audit and operations without turning the metadata plane into a second sensitive-data store.

Challenges we ran into

The hardest challenge was proving a complete causal chain instead of assembling adjacent demos. A receipt, an invalidation finding, and a replay are not meaningful if they cannot be shown to describe the same decision. We therefore carried content-addressed identities through every boundary and added direct readback at each external effect.

Stable DataHub Core also does not yet expose every Agent Registry primitive shown in newer client documentation. Rather than quietly depending on a release-candidate server, we built typed Document compatibility projections with explicit migration identifiers, documented the gap, and prepared an RFC path for native metadata.

Distributed failure handling was another major challenge. We tested crashes before and after external effects, lease expiry, redelivery, concurrent workers, and uncertain completion. The system distinguishes logical idempotency from physical writes and refuses schema drift or corrupted persisted state.

Finally, completeness is asymmetric: one matching dependency can prove a decision was affected, but proving it was unaffected requires complete coverage. GlassBox tracks dependency resolution and field-lineage coverage separately so it never turns missing evidence into false certainty.

Accomplishments that we're proud of

  • A real end-to-end DataHub integration, not a static dashboard or replay fixture.
  • Canonical signed receipts with independently verifiable integrity.
  • Deterministic, versioned materiality decisions with durable reason codes.
  • DataHub writeback for receipt projections, incidents, and verified closure.
  • Read-only forensics through both MCP and a reusable DataHub Skill.
  • Authorized, isolated recovery with append-only supersession.
  • PostgreSQL crash recovery and at-least-once delivery proofs.
  • Reproducible Python wheel and source builds with checksums and a CycloneDX SBOM.
  • A one-command flagship estate that boots a pinned DataHub Core environment and proves the complete causal chain.
  • A benchmark and ablation harness that demonstrates why dependency identity, field lineage, deterministic policy, and trust verification each matter.

What we learned

The key lesson is that provenance becomes operational only when it can drive a decision.

Capturing more logs is not enough. Evidence needs stable identities, explicit trust, honest completeness, deterministic policy, and a reversible workflow. DataHub is the natural substrate because it already knows what assets mean and how they relate. GlassBox turns that governed knowledge into a control loop for AI decisions.

We also learned that idempotency must be proven at every boundary. Returning the same receipt ID is useful; proving zero duplicate state, fresh readback, and safe recovery after a crash is what makes it dependable.

What's next for GlassBox

Next, we want to work with DataHub maintainers on the native agent-decision metadata model, land the portable forensics Skill, and publish the Action and MCP packages through trusted release infrastructure.

Product work will focus on organization-scoped onboarding, policy authoring, scheduled re-verification, richer field-level influence, and recovery approvals integrated with existing incident and ownership workflows.

The north star remains the same: make every consequential agent decision explainable, change-aware, and safely recoverable inside the DataHub ecosystem.

Built With

Share this project:

Updates