Nasl3yn

Cinematic Context Platform: AI agent giving video pipelines full DataHub context (lineage, MCP server, autonomous orchestration)

Comment

Nasl3yn DataHub: Cinematic Context Platform

Inspiration

Nasl3yn (نصلعين) is my cinematic AI platform generating documentary films from Arabic civilizational narratives. The current film, نصلعين (24 scenes across 6 days), traces humanity's arc from primitive tools → AI → sovereign covenant. Each scene is a generated video asset with rich metadata: prompt, model, resolution, seed, cost, lineage.

The problem: Video generation pipelines are black boxes. No lineage. No schema. No context for agents.

The insight: DataHub's open-source Context Platform (MCP Server, Agent Context Kit, DataHub Skills) is exactly what cinematic AI needs — a knowledge graph where every asset knows its ancestors, every pipeline writes back context, and agents can reason over the graph.

What I Learned

Area Insight
DataHub MCP Server Exposing cinematic operations as MCP tools lets any agent (Claude, Cursor, custom) orchestrate video pipelines without custom integrations
DataHub Skills Reusable, versioned units (create_video_asset, trace_lineage, register_pipeline) make the platform extensible — new skills = new capabilities without core changes
ML Lineage DataHub's end-to-end ML lineage (training data → features → model → deployment) maps perfectly to cinematic lineage (prompt → model → render → encode → asset)
Apache 2.0 Open-source license enables community contributions — critical for hackathon judging criteria

How I Built It

Architecture

graph LR
    A[Nasl3yn Agent] --> B[MCP Server]
    B --> C[DataHub Graph]
    C --> D[Cinematic Assets]
    C --> E[Pipeline Runs]
    C --> F[ML Models]

Core Components

Component Tech Purpose
DataHub Client httpx + GraphQL Async GraphQL/REST wrapper with retry logic
MCP Server mcp Python SDK 8 tools: create_video_asset, trace_lineage, register_pipeline, search_assets, update_pipeline_status, get_ml_lineage, add_lineage_edge, get_video_asset
DataHub Skills 7 skill classes Reusable units agents invoke via MCP: asset CRUD, lineage, pipeline, metadata validation
Orchestrator Agent asyncio + dependency resolution Runs workflows (single-scene, multi-scene, lineage) with parallel execution + semaphore
Cinematic Schema Pydantic models VideoAsset (prompt, model, resolution, duration, codec, seed, cost, tags), PipelineRun (DAG YAML, status, I/O assets)

Workflow Templates

Single Scene:

orchestrator.create_single_scene_workflow(
    scene_name="desert_dawn",
    prompt="Cinematic desert dawn, golden hour, 8k",
    model="veo-2",
    resolution="3840x2160",
    duration_seconds=8.0,
)

Multi-Scene (Nasl3yn Covenant — 6 scenes):

orchestrator.create_multi_scene_workflow(
    project_name="nasl3yn_covenant",
    scenes=[{"name": "day_1", "prompt": "..."}, ..., {"name": "day_6", "prompt": "..."}],
)

Lineage Establishment:

orchestrator.create_lineage_workflow(
    prompt_urn="urn:li:dataset:...prompt_nasl3yn_covenant...",
    asset_urns=[day_1_urn, day_2_urn, ..., day_6_urn],
)

Example Output (Judge-Ready)

{
  "urn": "urn:li:dataset:(urn:li:dataPlatform:nasl3yn,nasl3yn_covenant_day_6_render,PROD)",
  "name": "nasl3yn_covenant_day_6_render",
  "customProperties": {
    "prompt": "Great covenant tree: QSR roots, MineAIder trunk, HADA knowledge branch, ZAKA business branch, data streams forming calligraphic Arabic letters",
    "model": "veo-2",
    "resolution": "3840x2160",
    "duration_seconds": "8.0",
    "codec": "hevc",
    "seed": "1234567890",
    "generation_time_ms": "52000",
    "cost_usd": "0.18",
    "tags": "covenant,tree,QSR,MineAIder,HADA,ZAKA,arabic,calligraphy,finale"
  }
}

Challenges Faced

1. DataHub MCP Server API Flux

The MCP Python SDK evolved rapidly during development. Tool registration patterns changed between versions. Solution: Pinned mcp>=1.0 in pyproject.toml, wrote thin adapter layer in server.py to isolate SDK changes.

2. GraphQL Schema Introspection

DataHub's GraphQL schema is massive (10k+ lines). Writing correct mutations for entity upsert, lineage edges, ML lineage required trial-and-error. Solution: Used DataHub's GraphiQL explorer interactively, captured working queries/mutations, embedded as raw strings in client.py.

3. Async Context Manager in CLI

Typer commands are sync; orchestrator needs async with for DataHub client lifecycle. Solution: Helper _run_async(coro) bridges sync CLI → async execution without nesting asyncio.run().

4. uv Venv + Editable Install

uv creates venv without pip binary; editable install installs to system site-packages by default. Solution: Used /home/ahm/nasl3yn-datahub-hackathon/.venv/bin/python -m pip install -e . explicitly.

5. License Detection on GitHub

GitHub didn't auto-detect Apache 2.0 from LICENSE file initially. Solution: Ensured exact Apache 2.0 header text; pushed to root; GitHub now shows "Apache-2.0" in license info.

Mathematical Foundation

The lineage graph forms a directed acyclic graph (DAG) $G = (V, E)$ where:

  • $V = {v_1, ..., v_n}$ are cinematic entities (prompts, models, renders, pipelines)
  • $E \subseteq V \times V$ are lineage edges $(u \to v)$ meaning "$u$ contributed to $v$"

For a multi-scene project with $k$ scenes, the lineage depth is:

$$ \text{depth}(G) = \max_{v \in V} \text{length}(\text{longest path to } v) $$

The orchestrator's parallel execution bound with semaphore $S$ (max concurrent workflows) gives theoretical speedup:

$$ T_{\text{parallel}} \approx \frac{T_{\text{serial}}}{S} + T_{\text{overhead}} $$

where $T_{\text{overhead}}$ includes DataHub API latency $\sim \mathcal{O}(|E|)$ for lineage writes.

Submission Checklist

  • [x] Public GitHub repo with Apache 2.0 license: https://github.com/ahm22004/nasl3yn-datahub
  • [x] Working code — all 9 tests pass
  • [x] Demo video (<3 min) — to record
  • [x] Devpost submission — to submit by Aug 10
  • [x] Track: Open / Wildcard (best fit for cinematic AI)

Try It

git clone https://github.com/ahm22004/nasl3yn-datahub
cd nasl3yn-datahub
uv venv && source .venv/bin/activate
pip install -e .
cp config.yaml.example config.yaml  # Add your DataHub credentials
python -m nasl3yn_datahub.agent.main list-skills-cmd
python -m nasl3yn_datahub.agent.main single-scene "desert_dawn" "Cinematic desert dawn, golden hour, 8k"

Built for the DataHub Agent Hackathon — where agents get context, and cinematic AI gets sovereignty. 🎬

Built With

  • agent-context-kit
  • agent-orchestration
  • apache-2.0
  • asyncio
  • cinematic-ai
  • datahub
  • datahub-skills
  • fastapi
  • graphql
  • hackathon
  • lineage-tracking
  • mcp-server
  • metadata-schema
  • open-source
  • pydantic
  • python
  • uv
  • veo-2
Share this project:

Updates