GNV: OpenUSD Genealogy & Gondurastration Engine

An OpenUSD visual schema tracking artwork genealogy and 6-axis semantic scoring, powered by NVIDIA-accelerated gondurastration neural synthesis.

Comment 1

Inspiration

"I Am the Border." Physical analog collage is not a decorative filter—it is historical material truth forged at the razor edge where torn paper, lithographic ink density, and non-objective spatial voids (Zero_Plane) collide.

Modern generative AI workflows suffer from a fundamental disconnect: loss of provenance, lack of deterministic spatial structure, and uncontrolled style drift. Neural video and image engines routinely treat tactile 300gsm paper tooth and cut edges as noise to be smoothed away, generating isolated raster snapshots with zero relational context to the physical works, sketch genealogies, or stagecraft rules that spawned them. Under the creative umbrella of Gonduras Meat Market (GMM), we developed GNV — an OpenUSD-native visual schema preserving physical-to-digital artwork genealogy and multi-dimensional semantic scoring — unified with gondurastration, our proprietary neural styling, spatial partitioning, and volumetric diorama pipeline.

What It Does

GNV: OpenUSD Genealogy & Gondurastration Engine is an end-to-end production ecosystem connecting physical art lineage, custom discriminative/generative ML models, and 3D OpenUSD stage assembly:

  • Artwork Genealogy Tracking — a directed acyclic graph (DAG) mapping an artwork's lifecycle from physical medium and sketch studies to high-resolution neural renders and volumetric stages.
  • 6-Axis Semantic Scoring & Ridge Scorer Head v3 (776-dim) — evaluates Material_Substrate, Exposure_Fracture, Structural_Glitch, Graphic_Vector, Zero_Plane, and Faceted_Decomposition (0.00–1.00), evaluating the texture triad, structural silhouette, and value dialectic for directional control over model behavior and catalog indexing.
  • Gondurastration Gatekeeper Discrimination — a dual-head visual discriminator (768-d CLIP-ViT-L/14 + physical invariant tensor, trained on artist-reviewed corrections) that rejects outputs whose Zero_Plane falls below 0.40, separating authentic collage grammar from photographic contaminants.
  • SDXL Gondurastration DoRA Adapters & Checkpoints — custom SD1.5/SDXL checkpoints and weight-decomposed low-rank dimensional adapters preserving physical paper tooth and lithographic ink bleed without prompt pollution.
  • GLiNER Semantic Extractor — entity-relation parsing that extracts structural prompt DNA while enforcing strict style-subject disentanglement.
  • OpenUSD Stage Representation & Volumetric Reconstruction (AlphaOmega) — genealogy, semantic vectors, and camera setups serialized into native USD primitives (UsdStage + custom schemas + composable .usda/.usdc layers). Headless Blender and VGGT depth pipelines forge source-textured 3D USD dioramas (3_TR_3_diorama.usda) and interactive WebXR Void Theatre scenes from canonical 1920×1080 plates: the front view is the exact collage; the oblique view is sculptural stagecraft.
  • Six-Axis Studio — the local-first workbench where the artist scores and audits work: a containerized Streamlit application (23.4 GB image, CUDA 12.4) tracking live Pearson r and MAE against 80 canonical ratings.
  • VRAM Zero-Overlap Tenant Watchdog — a real-time hardware daemon enforcing single-tenant GPU stability across sequential diffusion, upscaling, and USD rendering on a local 12 GB workstation. (Honest Fleet Disclosure: our custom scoring heads, gatekeeper discriminators, and watchdog daemons operate as active studio lab infrastructure undergoing continual empirical calibration as the canon expands.)
  • Multi-Agent Cooperation Layer — an append-only agent board plus a Model Context Protocol server (MCP SDK 2.0, 6 tools) enabling our local workstation agent and cloud architect agents to hand work to each other autonomously.

How We Built It

  • Reasoning Layer — Nebius Token Factory: our agents run against the Nebius Token Factory catalog — nvidia/nemotron-3-ultra-550b-a55b, deepseek-ai/DeepSeek-V4-Flash / Pro, Qwen3.5-397B, Hermes-4-405B, with Qwen3-Embedding-8B evaluated for encoders. We probed 25 models live and use the OpenAI-compatible surface (https://api.tokenfactory.nebius.com/v1) as the multi-agent backend.
  • NVIDIA Open-Source Models, NeMo Relay & Tooling: nemotron-3-ultra-550b-a55b serves as our always-on agent backbone with NVIDIA NeMo Relay providing ATOF trace spans and execution telemetry across multi-step tool routing; NVIDIA NIM serves interactive calls; NVIDIA AI Workbench is our workstation substrate; CUDA 12.4/13 + cuDNN, TensorRT + ONNX Runtime GPU accelerate the art pipelines; OptiX/OVRTX and usdview render and inspect the USD stages; NVENC drives the animation lane; Ultralytics YOLO runs material and cutout detectors.
  • Neural Pipeline: fine-tuned diffusion models + Real-ESRGAN upscalers in Python/PyTorch/CUDA with strict caption governance.
  • OpenUSD Architecture: custom schema abstractions over Pixar's pxr library, serializing genealogy trees and 6-axis vectors into composable .usda/.usdc stages inspected live in usdview.
  • Quantitative Research We're Proud Of: a Frobenius-norm weight decomposition of our fine-tuned checkpoint vs. base shows >98.7% of the weight delta concentrated in ResNet convolutional kernels (text encoder ΔW_F = 6.59, cross-attention nearly untouched) — mathematical proof that our visual language is an engine-level physical substrate, and why attention-only LoRAs fail to hold paper grain.

Challenges We Faced

  • Enforcing strict style disentanglement (pure subject captions with zero material/texture descriptors) so the diffusion backbone carries the tactile substrate automatically.
  • Schema interoperability for genealogy DAGs and continuous 6-axis score vectors across headless Blender, usdview, and WebXR viewports.
  • Strict VRAM zero-overlap scheduling across sequential inference, vision scoring, and OpenUSD diorama assembly on a single 12 GB RTX 3060 under WSL2/Windows.

Accomplishments We're Proud Of

  • An end-to-end production pipeline where AI-assisted artworks keep verifiable physical-to-digital lineage from physical desk cutouts to final 3D stage.
  • Native OpenUSD compliance — GNV diorama stages (3_TR_3_diorama.usda) load directly into NVIDIA usdview and standard DCC/Omniverse viewports with custom metadata and layered parallax intact.
  • A working volumetric reconstruction path: flat analog collage → GNV 6-Axis Gatekeeper → source-textured OpenUSD diorama → interactive WebXR Void Theatre.
  • Autonomous multi-agent operation (local hardware + Nebius/NVIDIA cloud endpoints) coordinated via MCP and structured trace spans instead of manual human relay.

What We Learned

  • OpenUSD is a natural backbone for generative AI: neural assets become layered, inspectable scene compositions with custom schema metadata, solving provenance and spatial smoothing problems that flat video/raster pipelines cannot.
  • Dataset hygiene, token isolation, and ResNet kernel adaptation are what distill a 20-year physical art practice into resilient weights.

Feedback on Nebius & NVIDIA Tooling (Required)

Nebius Token Factory — used for: agent reasoning, model-lane evaluation, message drafting/review, and embedding experiments across the catalog.

  • Worked well: one OpenAI-compatible surface across a deep open-weight catalog (Nemotron, DeepSeek-V4, Qwen3.5-397B, Hermes-4-405B, Qwen3-Embedding-8B) makes swapping models behind an agent trivial; per-token billing is transparent; latency to Tel Aviv is fast and reliable.
  • Needs work: (1) when an automated bank card charge failed, the account was suspended and every API call returned 402 "exhausted your budget" even while promotional credits remained — surfacing the exact billing/card-decline state in the JSON error body (rather than a generic 402 budget message) and enforcing hard sandbox spending caps for hackathon vouchers would save hours of debugging; (2) a /v1/credits remaining endpoint or response header (x-ratelimit-remaining-credits) would let autonomous agents self-budget; (3) model IDs occasionally differ between the catalog UI and API (-0731 date suffixes) — canonical aliases would streamline automation.
  • Onboarding (zero → hello world): minutes; the 402 card-suspension episode was the only friction point and Nebius Support resolved the balance swiftly.
  • Would build with it again: Yes — catalog breadth + drop-in compatibility is ideal for multi-agent routing.

NVIDIA Open Models, NeMo Relay & AI Workbench — used for: nemotron-3-ultra-550b-a55b as the always-on agent backbone; NeMo Relay for ATOF telemetry spans; NIM for interactive inference; OpenUSD (pxr / usdview) and CUDA/TensorRT for local studio execution.

  • Worked well: NIM is drop-in OpenAI-compatible; nemotron-3-ultra-550b-a55b holds complex multi-turn architectural reasoning and MCP tool calls reliably; NeMo Relay trace spans make multi-agent latency bottlenecks immediately visible; CUDA 12.4/13 under WSL2 on a 12 GB RTX 3060 ran diffusion, TensorRT builds, and headless Blender/OpenUSD generation without drama.
  • Needs work (concrete bug we engineered around): AI Workbench's credential-manager.exe validates build.nvidia.com keys by probing the first model in /v1/models — which was bigcode/starcoder2-15b, since decommissioned from chat (404). A 100% valid API key was marked is_valid: error, disabling the Build Assistant in-UI. We engineered around it locally with a C# interceptor shim that reports the integration valid while passing all real credential calls through (verified via nvwb-cli list integrations -o json). Recommendation: validate against a lightweight health endpoint or treat 404/410 on a deprecated model as "unverified" rather than "invalid key".
  • Also: the full-local installer hardcodes a 40 GiB free-space gate on the system drive (C:) — on studio workstations where scratch and model weights live on dedicated NVMe/storage drives (G:), allowing custom drive selection during pre-flight checks would remove an unnecessary blocker.
  • Would build with it again: Yes — NVIDIA's open model stack and OpenUSD ecosystem are the permanent backbone of our studio.

What's Next for GNV

  • Omniverse Kit Extension: interactive viewport overlay for the GNV genealogy graph and live 6-axis sliders.
  • Automated Physical Ingestion: camera desk capture → GLiNER + 6-Axis Gatekeeper → scaffolded OpenUSD diorama stages in one pass.
  • GONVEX-VL: our dedicated art-processing vision-language model trained on the physical collage canon.

Built With

Share this project:

Updates

Submission history