ApexMarket: Verified Agentic Compute & Inference Arbitrage

• 

  Autonomous batch GPU arbitrage anchored by Nebius Token Factory & NVIDIA Nemotron-3-Nano with 32.18% empirical cost savings.

• 

  Zero-overhead telemetry: 898 tok/s swarm velocity, 52ms p50 TTFT, and sub-10ms 5-tier cryptographic audit verification delay.

• 

  Capacity-aware DAG routing batch shreds across Nebius Authoritative Anchor, RunPod H100 SXM5, and Community L40S spot swarms.

• 

  Zero-trust audit portal (/verify): Independent Web Crypto SHA-256 validation of signed attestation certificates with zero backend.

• 

  Automated fraud mitigation: Detecting rogue INT4 quantization drift, slashing payout to $0.00, and failing over to Nebius.

• 

  Fail-closed Confidential Air-Gap: 100% of sensitive batch payloads route directly to Nebius Token Factory with zero spot exposure.

• 

  Forensic Attestation Drawer: Itemized 5-tier inspection displaying byte tokens, HMAC canary signatures, and Nemotron logprob audit.

Inspiration

Enterprise AI batch workloads (financial report processing, document classification, code translation, and tabular data extraction) represent the fastest-growing segment of corporate LLM spending. While dynamic GPU spot capacity offers a 40% to 60% cost reduction, enterprises are forced to pay retail hyperscaler rates due to three foundational risks:

1. Silent Quantization Degradation: Unannounced INT4/FP8 weight truncation or unauthorized model substitution by third-party spot hosts.
2. Execution Volatility: Node evictions, dropped batches, and incomplete inference streams.
3. Absence of Proof Before Payment: Traditional cloud billing forces enterprises to pay for compute before validating token integrity.

Existing routers (such as OpenRouter) rank providers by price floor and aggregated uptime, but they cannot cryptographically verify completion integrity or protect workloads from model drift. We built ApexMarket to introduce proof-before-payment compute arbitrage: an autonomous orchestration and verification clearinghouse that distributes batch workloads across cost-effective spot nodes, audits every output against an authoritative Nebius Token Factory anchor, and conditionally settles micro-dollar balances.

What It Does

• Capacity-Aware Greedy Shredding: Ingests batch workloads (up to 5,000 prompts) and partitions them into micro-batch shreds scheduled across verified spot nodes based on real-time price-to-throughput rankings.
• Fail-Closed Confidential Air-Gap: Sensitive enterprise batches default to CONFIDENTIAL, isolating 100% of execution to the SOC-2 compliant Nebius Token Factory Anchor. Non-sensitive workloads (PUBLIC or SYNTHETIC) unlock dynamic multi-node spot arbitrage.
• 5-Tier Forensic Verification Pipeline:
  1. Tier 1 (Authoritative Tokenizer Billing): Computes exact HuggingFace byte token counts to prevent inflated billing deltas.
  2. Tier 2 (Rotated Canary Probe Challenge): Injects 4% dynamic cryptographic challenge prompts sealed with HMAC-SHA256 signatures to detect output caching and model substitution.
  3. Tier 3 (Secret Anchor Sampling): Routes a 5% CSPRNG sample to the unquantized Nebius Anchor at temperature 0.0 to establish a golden cosine baseline (≥ 0.985).
  4. Tier 4 (Teacher-Forcing Logprob Scoring): Evaluates output tokens against Nebius anchor weights using Negative Log-Likelihood (NLL). An anomalous divergence (NLL > 2.80) flags INT4 quantization drift.
  5. Tier 5 (NVIDIA Nemotron Semantic Arbiter): Resolves borderline outputs (2.50 ≤ NLL ≤ 2.80) using Nemotron-3-Nano as an automated LLM judge.
• Automated Sashing & Instant Failover: If a spot node fails canary validation or suffers INT4 drift, its escrow payout is slashed to $0.00, the node is quarantined, and all remaining shreds automatically re-route to Nebius Token Factory.
• Independent Attestation Verification Portal (/verify): Produces tamper-evident signed JSON certificates (apexmarket-attestation-{job_id}.json). Anyone can drag-and-drop the certificate into our client-side Web Crypto verifier to validate SHA-256 digests independently.
• Double-Entry Micro-Dollar Settlement: Enforces mathematical balance conservation ($Total = Available + Escrow + Probation + Withdrawable + PlatformFee$) with automated payout distribution via PayPal REST APIs.

How We Built It

• Authoritative Anchor Infrastructure: Deployed nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B on Nebius Token Factory via serverless endpoints (api.tokenfactory.us-central1.nebius.com/v1) for reference sampling, teacher-forcing scoring, and semantic arbitration.
• Backend Architecture: Built with FastAPI, Pydantic v2, and SQLAlchemy. Implemented capacity-aware water-filling schedulers, offline tokenizer engines, and HMAC-SHA256 canary generators.
• Frontend & Visual Topology: Next.js 14 (App Router) with Tailwind CSS, React Flow visual DAG orchestration, SVG real-time telemetry gauges (p50/p95/p99 TTFT, token velocity), and sliding attestation inspection drawers.
• Live Empirical Benchmarking: Validated through automated test harnesses hitting Nebius Token Factory:
  • 32.18% Verified Net Cost Savings ($0.000923 vs. $0.001361 baseline per test suite).
  • 885.9 tokens/sec sustained swarm throughput.
  • 52.0ms p95 Time-To-First-Token (TTFT) on Nebius Nemotron endpoints.

Challenges We Overcame

1. Handling Thought Isolation in Reasoning Streams: Processing <think>...</think> tokens during streaming TTFT measurements required building dedicated streaming regex parsers so reasoning tokens did not skew user-facing latency.
2. Cross-Provider Tokenizer Drift: Preventing micro-dollar billing mismatches across disparate spot endpoints by enforcing authoritative offline HuggingFace tokenization directly on the proxy orchestration layer.
3. Designing a Cheater-Proof Penalty Function: Constructing an economic game-theoretic penalty curve where dishonest spot operators face a mathematically negative expected return (E[cheat] ≈ -$59.26 ≪ 0).

Documented Limitations & Edge Cases

• Statistical Significance Threshold: Negative Log-Likelihood (NLL) divergence scoring requires a minimum batch size of 5 prompts to establish statistical confidence.
• Node Probation Window: New unvetted spot nodes operate under a 7-day probation period with a 5% volume cap and mandatory escrow holdback.
• PayPal Settlement Granularity: PayPal Payouts enforce a $0.01 minimum transaction floor; sub-cent fractional balances are tracked as micro-dollar credits and rolled over to subsequent batches.

What's Next

• Integrating NVIDIA Confidential Computing (CC) on H100/H200 instances for cryptographic hardware-level attestation.
• Adding streaming token-level logprob arrays as Nebius expands Token Factory batch endpoint capabilities.

Built With

• docker
• fastapi
• huggingface-tokenizers
• nebius-token-factory
• next.js-14
• nvidia-nemotron
• paypal-rest-api
• pydantic
• pytest
• python
• react-flow
• sqlalchemy
• tailwind-css
• typescript
• vercel
• web-crypto-api