PatchTree

Speculative sandbox forking engine using NVIDIA Nemotron on Nebius Token Factory to explore, verify, and merge autonomous Python bug fixes with zero regressions.

Comment

PatchTree — Speculative Sandbox Forking & Multi-Stage Verification for Autonomous Code Repair

Live Application: patch-tree.vercel.app Source Code: github.com/icodesAS/PatchTree (Open Source under MPL-2.0) Track: Coding and Agentic Engineering Track

💡 Inspiration

Single-shot coding agents fail at software engineering for one simple reason: they make a single guess, run the named test once, and pray. If the patch breaks concurrent execution, weakens assertions, or passes by coincidence due to test flakiness, single-shot agents merge anyway. That isn't engineering—that's a coin flip.

We built PatchTree to turn autonomous code repair into an empirical, speculative search engine. Instead of a linear loop, PatchTree forks clean sandbox environments at branch points, explores multiple candidate fixes in parallel across tiered NVIDIA Nemotron models, and subjects passing candidates to a rigorous 3-stage verification gauntlet before declaring success.

⚡ What It Does

When a failing Python repository and test suite are submitted:

  1. AST Context Isolation: Extracts and trims only the exact failing functions, call graphs, and local scope down into a compact AST context (<3KB prompt payload).
  2. Speculative Sandbox Forking: Spawns parallel sandbox forks using Nebius Contree MicroVM sandboxes.
  3. Dynamic Model Routing:
    • Nano 30B: Generates fast, surgical syntax/off-by-one repairs ($0.003 / patch, ~12s).
    • Super 120B: Dispatches targeted architectural and state-management repairs.
    • Ultra 495B (Deep Reasoning): Only activated if lightweight branches fail. Ingests full execution tracebacks and AST call-graphs to resolve subtle semantic regressions.
  4. 3-Stage Empirical Verification:
    • Target Test Pass: Did the failing unit test turn green?
    • Flaky-Test Stability (5x Re-run): Re-runs the test suite 5 consecutive times across fresh forks to detect race conditions and nondeterministic ordering.
    • Adversarial Regression Suite: Synthesizes edge-case inputs (nulls, mixed casing, empty collections) to ensure the patch didn't introduce regressions.
  5. Mission Control Canvas: Visualizes the entire search DAG in real-time with React Flow, live terminal execution logs, patch diffs, and exact token/cost metrics.

🛠️ How We Built It

  • Backend Orchestrator: Python + FastAPI streaming DAG state changes over REST & Server-Sent Events to Supabase PostgreSQL.
  • LLM Engine: Nebius Token Factory hosting NVIDIA Nemotron 3 open-weight models (nvidia/nemotron-3-nano-30b, nvidia/nemotron-3-super-120b, nvidia/nemotron-3-ultra-495b).
  • Sandboxing & Isolation: Nebius Contree SDK running lightweight containerized MicroVM forks with sub-second rollback capabilities.
  • Frontend Mission Control: React 19, TypeScript, Vite, TailwindCSS, and React Flow.

📊 Real Benchmark Results

PatchTree was tested against three real-world production defects:

  1. noshaeec/checkout-pricing-suite: Fixed a promotional pricing off-by-one boundary defect in 58.6s for $0.0086.
  2. GHOST4709/BackendTestRun: Repaired an unhandled empty-workspace crash in 69.5s for $0.0071.
  3. GHOST4709/drivelegal: Diagnosed and repaired a case-insensitive query defect in 90.1s for $0.0075 with a 1-line surgical fix: func.lower(TrafficLaw.city) == func.lower(city).

🧗 Challenges We Ran Into

  • CRLF vs. LF Line Endings: Windows-formatted repositories caused standard git apply to fail with Hunk #1 FAILED (different line endings). We implemented automatic AST normalization and whitespace-agnostic patch verification.
  • Context Window Blindness: Microservices like FastAPI returning clean HTTP 404s without Python tracebacks blinded LLM root-cause analyzers. We augmented prompt payloads with AST endpoint route mapping to guarantee full coverage.

🏆 Accomplishments That We're Proud Of

  • Demonstrating sub-100 millisecond sandbox state rollbacks for speculative exploration.
  • Achieving 100% test pass rate across all three public production benchmarks for less than $0.01 per run.
  • Zero hallucinated imports or unnecessary multi-file refactors: fixes are tight, 1-to-5 line surgical diffs.

🚀 How We Used Nebius Token Factory & NVIDIA Nemotron

  • Cost-Optimized Model Routing: Nano 30B handles 80% of candidate exploration, drastically reducing token consumption. Ultra 495B reasoning is reserved strictly for high-entropy root-cause diagnoses.
  • Contree Sandbox Forking: Instant filesystem snapshotting allowed us to test 4 competing hypothesis branches simultaneously without disk contention.

💬 Feedback on Nebius & NVIDIA Tools (Mandatory Hackathon Section)

  • NVIDIA Nemotron: Outstanding instruction adherence when producing strict unified diff patches without markdown chatter. Ultra 495B's chain-of-thought capability for tracebacks is exceptional.
  • Nebius Token Factory: Fast time-to-first-token (TTFT) and high concurrency made speculative parallel branch execution feasible without latency bottlenecks.
  • Suggestion: Adding native snapshot-diff APIs directly in the Contree SDK would make sandbox state inspection even more seamless.

🔮 What's Next for PatchTree

  • Polyglot AST Support: Adding Tree-sitter parsers for TypeScript, Go, and Rust.
  • GitHub Action: Autonomous patchtree/verify bot that comments directly on pull requests with verifiable diffs.
  • Synthetic RL Engine: Using verified patch trees to produce fine-tuning datasets for smaller local models.

Built With

Share this project:

Updates