RallyTrace

RallyTrace uses DataHub context and write-back to stop coaches from acting on compromised performance reports and route the exact data issue to its owner.

Comment 1

Inspiration

A polished sports dashboard can still be unsafe for the decision a coach is about to make. A number may look reasonable while the assumptions that created it — join grain, measurement completeness, lineage, and ownership — have disappeared.

I wanted to turn DataHub's context graph into an operational guardrail, not another catalog search or generic data-quality score.

What it does

RallyTrace asks one precise question:

Is this exact performance report safe for the exact coaching decision being made now?

The agent reads the report through the official DataHub MCP server, traces upstream lineage, inspects schema fields and quality context, resolves the accountable owner, and applies a deterministic decision contract.

It returns one of three outcomes:

  • APPROVED FOR THIS DECISION
  • REVIEW REQUIRED
  • HOLD — DO NOT USE

The result is written back to DataHub as a status tag and decision context, so the next analyst, coach, or agent inherits the finding.

The primary football scenario catches a silent join-grain failure. A readiness report lost session_id and joined on athlete_id + session_date, allowing morning recovery data to be combined with an evening team session. RallyTrace holds the report, routes the exact issue to its owner, then verifies the corrected pipeline and approves the report.

A second table-tennis scenario demonstrates that the policy core transfers beyond one sport: an incomplete measurement series triggers review instead of being treated as a valid comparison.

How I built it

RallyTrace is a Node.js browser application with a deterministic policy engine and two DataHub provider modes.

The live provider connects to the official DataHub MCP server and uses get_entities, list_schema_fields, get_lineage, add_tags, and update_description.

A reproducible Python SDK bootstrap creates nine synthetic DataHub datasets, eight lineage edges, schemas, owners, descriptions, and RallyTrace tags. No real athlete, federation, child, health, or proprietary records are used.

The public demo runs an explicitly labelled evidence-fixture provider so judges can test the complete decision flow without credentials. Both providers use the same agent orchestration and deterministic policy engine, and the exact official MCP read and mutation argument shapes are covered by contract tests.

Challenges I ran into

The core design challenge was avoiding a plausible AI explanation becoming an unreliable approval signal.

I separated authority from explanation: deterministic logic owns the decision status, while an explanatory layer can only communicate the evidence. A model failure can reduce personalization, but it cannot silently turn compromised data into an approved coaching input.

The other challenge was making a metadata product understandable to a coach in seconds. The interface therefore starts with the decision to protect and shows only the evidence that changes whether the report can be used.

Accomplishments that I am proud of

  • DataHub is used for both context retrieval and graph write-back.
  • The football flow closes the loop from hold, to exact root cause, to verified recovery.
  • The table-tennis case proves the decision-contract pattern is reusable.
  • The public demo is safe, synthetic, credential-free, and honest about its evidence mode.
  • Eleven automated tests cover policy, orchestration, transferability, and exact MCP tool contracts.
  • The project includes an Apache 2.0 license, reproducible DataHub runbook, architecture document, and security boundary.

What I learned

Metadata becomes much more valuable when it is evaluated against an intended action. A dataset is not simply trusted or untrusted; it can be fit for one decision and unsafe for another.

I also learned that agent write-back matters as much as retrieval. The finding should not disappear into a private assistant transcript — it should become shared context in the graph.

What's next for RallyTrace

Next I would add configurable decision-contract templates, native DataHub assertion ingestion, notifications to the recorded owner, and additional sports packs for injury-risk screening, scouting comparisons, and training-load planning.

The broader opportunity is any domain where a person is about to act on a report and needs provenance, schema, quality, and accountability translated into a clear operational gate.

Disclosure

RallyTrace was created during the DataHub Agent Hackathon build period. I designed the product, decision contracts, sports scenarios, and reviewer experience; implemented and validated the project with OpenAI Codex; and personally reviewed the football and table-tennis domain logic. All demonstration data is synthetic.

Built With

Share this project:

Updates