Nutri

We consume food daily, why not treat data the same? Nutri unifies freshness, health, and compliance into standardized Nutrition Labels with instant visibility and AI-powered, automated remediation.

Comment

Nutri

My inspiration for Nutri

When shopping for food at the grocery store, people rarely read complex biochemical lab reports or manufacturing logs. Instead, they look at an FDA Nutrition Facts panel that condenses calories, macro-nutrients, daily percentage values, and allergen warnings into a clear, standardized label.

Data engineers, analysts, and AI agents face a similar problem when evaluating datasets in modern data platforms. Determining whether a dataset is safe to use requires asking several questions:

  • Is this table fresh or stale?
  • Are the columns properly documented?
  • Are data quality tests passing?
  • Is there an owner assigned if something breaks?

Checking all of this in DataHub usually means clicking through five different tabs for schema, lineage, quality assertions, ownership, and governance terms, then manually piecing the answer together.

Nutri was built to solve this problem. It generates a standardized Data Nutrition Facts label for any dataset, dbt model, or dashboard in DataHub, pairing a 0–100 Trust Score with clear sub-scores that anyone can evaluate in a few seconds.

What it does

Nutri transforms DataHub metadata into a standardized health label for catalog entities across platforms including Snowflake, dbt, PostgreSQL, Tableau, Looker, and PowerBI.

Key capabilities include:

  1. Deterministic Trust Score (0–100): Computes a reproducible score using four weighted sub-scores (% Daily Value):

    • Freshness: Evaluates age decay against expected refresh cadence and staleness limits.
    • Completeness: Measures column documentation coverage along with ownership, domain, and glossary term presence.
    • Lineage Depth: Checks upstream and downstream edge connectivity and cross-platform lineage diversity.
    • Test Coverage: Tracks the presence and historical pass rates of DataHub data quality assertions.

  2. LLM Verdict and Anti-Hallucination Guardrails: Powered by Google Gemini 2.5 Flash (@google/genai), Nutri generates a one-sentence summary highlighting dataset strengths and score bottlenecks. Undocumented columns are explicitly marked as Undocumented / Needs Description rather than inventing false descriptions.

  3. Warning Banner: Automatically flags critical risk factors such as failing assertions, unowned assets, PII tags, or stale data in a visual callout banner.

  4. DataHub Write-Back Pipeline: Sends GraphQL mutations to DataHub GMS to store five structured properties (nutriTrustScore, nutriFreshnessScore, nutriCompletenessScore, nutriLineageScore, nutriTestCoverageScore) and applies the nutri:needs-attention tag whenever a score falls below 70.

  5. Flexible Interfaces:

    • Interactive Web App: Next.js catalog explorer with real-time scoring weight customization.
    • Embedded UI Mode: Lightweight card designed for embedding inside external catalogs or internal documentation.
    • DataHub MCP Server: Exposes Model Context Protocol tools for AI coding assistants.
    • CLI Tool and CI/CD Quality Gate: Enables automated score checks in terminal environments and build pipelines.
    • Talk-to-Data SQL Analytics Agent: AI agent that verifies data health before generating schema-grounded SQL queries.

DataHub Technologies Used

Nutri leverages multiple components of the DataHub platform ecosystem:

  1. DataHub OSS / Core Platform: Integrates directly with DataHub GMS via GraphQL (/api/graphql) to fetch catalog metadata, assertion run events, glossary terms, and lineage graphs. It performs bi-directional sync by writing back 5 structured properties (nutriTrustScore, nutriFreshnessScore, nutriCompletenessScore, nutriLineageScore, nutriTestCoverageScore) and auto-tagging entities with nutri:needs-attention.

  2. DataHub MCP Server: Implements a standalone Model Context Protocol server (scripts/mcp-server.ts) exposing tools like get_data_nutrition_facts, explain_score, write_trust_score, and generate_remediation_patch for AI tools and coding assistants.

  3. DataHub Agent Context Kit: Features a context kit module (src/services/context-kit.ts) that packages schema definitions, quality warnings, score breakdowns, and governance rules into prompt context packets for LLMs.

  4. DataHub Skills: Packaged custom agent instructions in .agents/skills/data-nutrition-facts/SKILL.md that allow AI coding assistants to evaluate dataset health, run CI/CD quality gate audits, and output dbt schema remediation patches.

  5. Analytics Agent: Built the Nutri Analytics Agent and Talk-to-Data Engine (src/services/agent.ts) that provides automated metadata diagnosis, SQL query generation grounded in schema definitions, and data quality risk banners.

How I built it

Nutri was built as a full-stack metadata platform and AI agent toolkit:

Challenges I ran into

  1. Designing a Fair Scoring Model:

    • Challenge: Building a scoring system that reflects dataset quality fairly without favoring a specific data warehouse or data platform.
    • Solution: Structured the model around four balanced pillars (Freshness, Completeness, Lineage, and Test Coverage) normalized from 0 to 100%. Externalized weight configurations so teams can tailor the scoring math to their specific priorities.

  2. Preventing LLM Hallucinations in Schema Documentation:

    • Challenge: Large language models tend to invent plausible descriptions when column metadata is missing.
    • Solution: Added strict prompt guardrails in DataHubContextKit that force the model to label missing metadata as Undocumented / Needs Description, keeping the generated summaries accurate.

  3. Traversing Cross-Platform Lineage Graphs:

    • Challenge: Catalogs span heterogenous entities from Snowflake tables to dbt transformation models and BI dashboards, each with different metadata shapes.
    • Solution: Wrote graph traversal logic over GraphQL that calculates lineage depth and platform diversity across connected nodes.

  4. Handling GraphQL Synchronization Gracefully:

    • Challenge: Keeping live DataHub GMS write-backs responsive without causing UI latency or failing during offline demonstrations.
    • Solution: Combined optimistic UI updates with a fallback catalog dataset (showcase-catalog.ts), allowing the app to work seamlessly in both live and offline modes.

Accomplishments that I'm proud of

  • Fully Auditable Scoring Engine: The scoring logic is completely rules-based and open source, so anyone can inspect the exact formulas in the codebase.
  • Bi-Directional DataHub Integration: Nutri does not just read catalog data; it writes back structured properties and tags low-trust assets with nutri:needs-attention directly in DataHub GMS.
  • Native AI Agent Integration: Exposed Nutri functionality through MCP tools, CLI quality gates, and agent skills so downstream AI code generators can check data health before running queries.
  • Production Web Deployment: Delivered a responsive Next.js web application deployed on Vercel, supporting standalone browsing and embedded card views.

What I learned

  • DataHub GMS Architecture: Gained a deeper understanding of DataHub GraphQL schemas, URN conventions, ownership aspects, assertion events, and structured property mutations.
  • Effective Metadata Context for AI: Learned how supplying structured metadata context via DataHubContextKit eliminates SQL generation hallucinations and improves summary quality.
  • Building MCP Servers: Explored how the Model Context Protocol standard enables AI tools like Claude, Gemini, and coding agents to interact directly with enterprise data catalogs.

What's next for Nutri

  • Automated dbt Remediation Pull Requests: Generating 1-click GitHub Pull Requests containing dbt schema.yml patches to fill in missing column descriptions and tests.
  • Lineage Degradation Tracing: Adding step-by-step lineage views that highlight exactly where a trust score drops along a pipeline from warehouse to dashboard.
  • Event-Driven Webhook Triggers: Re-evaluating scores automatically when DataHub receives new schema or quality ingestion events via Kafka or webhooks.
  • Domain and Team Scorecards: Aggregating nutrition facts across entire business domains or engineering teams to monitor overall data governance health.

Built With

  • datahub-gms
  • dbt
  • docker
  • git
  • gmi-cloud
  • graphql-api
  • mcp
  • rest
  • snowflake
  • vercel
  • vitest
Share this project:

Updates