• 

• 

Analyzed README.md:L101-283 Ran check docs directory Analyzed ENGINE_ARCHITECTURE_PART2.md:L1-100 Analyzed ENGINE_ARCHITECTURE_PART2.md:L101-150

Here is a complete, publication-ready "About the Project" write-up tailored for your Devpost hackathon submission, formatted in Markdown with LaTeX math support.

---

About PatchPilot

From Failing Tests to Verified Fixes

---

💡 Inspiration

Debugging is easily one of the most cognitively taxing and expensive phases of software engineering. Every developer knows the routine: a test fails in CI, and you spend hours context-switching between cryptic stack traces, reproducing the bug locally, reading through unfamiliar code, guessing root causes, tweaking logic, and waiting for test suites to re-run.

While modern AI coding assistants can generate code snippets on demand, they do not run code, reproduce errors, or verify their suggestions. They hallucinate plausible-looking patches that break existing functionality, leaving the burden of verification entirely on the developer.

We asked ourselves:
What if an AI engineer didn't just guess code, but actually reproduced the bug in an isolated sandbox, diagnosed the root cause with deep reasoning, wrote a minimal unified diff, and proved that every test passed before asking you to merge?

That was the spark behind PatchPilot—an autonomous, self-healing debugging engineer powered by NVIDIA Nemotron and Nebius Token Factory, built to take repositories from failing tests to verified fixes with mathematical certainty and zero guesswork.

---

⚙️ What PatchPilot Does

PatchPilot takes a codebase and a bug description (or failing test reference), autonomously investigates the root cause, and delivers an evidence-backed repair:

1. Reconnaissance & Ingestion: Upload any repository archive (or point to a demo). PatchPilot indexes the codebase, discovers test frameworks (pytest, unittest), analyzes dependencies, and flags sensitive files (.env, secrets).
2. Deterministic Sandbox Reproduction: Before touching a single line of code, PatchPilot executes the baseline test suite inside a hardened, isolated Docker container to confirm the exact failure signature and execution state.
3. Evidence-Based Diagnosis: Armed with the baseline stack trace, bounded file excerpts, and repository context, NVIDIA Nemotron (Llama-3.1-Nemotron-70B-Instruct-HF) formulates ranked hypotheses and determines the minimal set of files requiring modification.
4. Minimal Patch Generation: The agent drafts a strict, standard unified diff (patch) targeting only the defect.
5. Human Approval Gate: Safety first. The developer reviews the unified diff side-by-side with syntax highlighting. Nothing is ever applied to production code without explicit human consent.
6. Execution Verification & Regression Audit: Upon approval, the patch is applied to a clean clone in an isolated sandbox. The test suite is re-run and compared against the baseline.
7. Evidence-Backed Repair Report: PatchPilot reports pass/fail metrics, test diffs, and execution timing—proving the fix worked without introducing regressions.

---

🔬 Formal Verification Model

Unlike static analysis or simple chat assistants, PatchPilot measures repair validity using closed-loop empirical verification:

$$V(P) = \begin{cases} \text{Verified} & \text{if } T_{\text{fail}}(P) = \emptyset \;\land\; T_{\text{pass}}(P) \supseteq T_{\text{pass}}(\text{baseline}) \ \text{Regression} & \text{if } \exists\, t \in T_{\text{pass}}(\text{baseline}) : t \notin T_{\text{pass}}(P) \ \text{Failed} & \text{otherwise} \end{cases}$$

Where:

• $T_{\text{fail}}(P)$ denotes the set of failing tests under proposed patch $P$.
• $T_{\text{pass}}(P)$ denotes the set of passing tests under proposed patch $P$.

A repair is marked verified if and only if every target failing test passes and zero regressions occur across the existing test suite.

---

🛠️ How We Built It

PatchPilot was engineered with a modular, security-conscious full-stack architecture:

1. The Autonomous Agent & LLM Layer

• Model: nvidia/Llama-3.1-Nemotron-70B-Instruct-HF hosted via Nebius Token Factory, queried with strict temperature calibration ($T = 0.05$ for patch generation, $T = 0.2$ for root-cause diagnosis).
• Agent Lifecycle State Machine: Built as a deterministic 7-stage engine (Reconnaissance $\to$ Baseline Execution $\to$ Diagnosis $\to$ Investigation Tools $\to$ Patch Generation $\to$ Human Approval $\to$ Verification).
• Tool Registry: Bounded, non-shell diagnostic tools (read_file_excerpt, search_source, inspect_manifest, run_approved_test_command) with strict per-job execution budgets.

2. Hardened Sandbox Execution

• Docker-based isolation with defense-in-depth security:
  • --network none: Complete network isolation preventing outbound data exfiltration or access to cloud metadata endpoints (169.254.169.254).
  • --user 65534:65534: Non-root execution (nobody:nogroup).
  • --cap-drop ALL: Strips all Linux kernel capabilities.
  • --read-only: Read-only root filesystem with ephemeral memory tmpfs (/tmp:rw,noexec,size=64m).
  • Resource caps: Memory bounded to $512\text{ MB}$, CPU throttling, and pids-limit 100 to prevent fork bombs.

3. Backend Architecture

• FastAPI (Python 3.11+ / 3.14): Fully typed endpoints using Pydantic v2 schemas and SQLAlchemy with SQLite for persistent job history and live step-by-step event logging.
• Fail-Closed Design: If a container runtime or dependency fails, PatchPilot flags an infrastructure failure rather than fabricating a false diagnostic.

4. Frontend Experience

• Next.js (App Router, TypeScript, Tailwind CSS): Modern developer cockpit featuring interactive unified diff viewers, live state machine progress trackers, failure classification badges, and a human-in-the-loop approval modal.

---

🧗 Challenges We Faced

1. Context Window vs. Repository Scale:
   Repositories can quickly overwhelm context windows. We implemented dynamic token budgeting: $$\sum_{i=1}^{k} \text{Size}(C_i) \le B_{\text{max}} \quad \text{where } B_{\text{max}} = 24\text{ KB}$$ By ignoring binaries, lockfiles, virtual environments, and caches, and selecting targeted AST/source slices based on the test trace, we fed Nemotron dense, highly relevant context.

2. Patch Precision and Whitespace Hallucination:
   LLMs frequently generate invalid unified diff offsets or hallucinate line numbers. We engineered a resilient patch validation and fuzzy application engine that verifies chunk headers (@@ -l,s +l,s @@), validates file paths against repository boundaries, and prohibits modification to protected files (.env, Dockerfile, .git).

3. Untrusted Code Execution in Sandboxes:
   Running code uploaded by users requires strict sandboxing. Preventing shell injection meant abandoning shell string interpolation entirely in favor of explicit argument arrays (["python", "-m", "pytest", "-v"]) executed inside constrained Docker containers.

4. Multi-Turn Repair Budgets:
   If a proposed patch fails verification, the agent enters a bounded self-correction loop ($\le 3$ iterations) feeding the new error back to the model without entering infinite loops or burning tokens.

---

🏆 Accomplishments That We're Proud Of

• True End-to-End Verification: We don't just "show" code—PatchPilot actually executes the code in Docker, catches failures, applies patches, and proves that 100% of tests pass.
• Sub-Second Nemotron Reasoning: Nebius Token Factory delivered blazing-fast inference speeds with nvidia/Llama-3.1-Nemotron-70B-Instruct-HF, allowing complex diagnostic hypotheses to be formed in seconds.
• Robust Security Posture: Zero network access during execution, non-root users, path traversal guards on every tool, and mandatory human approval before changes are merged.
• 32 Backend & 11 Frontend Automated Tests: The platform itself is thoroughly tested with comprehensive unit and integration test coverage.

---

📚 What We Learned

• NVIDIA Nemotron's Diagnostic Capabilities: Nemotron 70B excels at technical root-cause synthesis when given clean, unpolluted tracebacks and bounded file contexts. Its ability to follow strict JSON schema instructions made automated agent transitions reliable.
• Closed-Loop Feedback is Superior to One-Shot Prompting: Providing the model with real containerized test outputs produced significantly higher quality fixes than any zero-shot prompting strategy.
• Designing Fail-Closed Agent Architectures: In autonomous agent systems, admitting "cannot reproduce due to missing dependencies" is infinitely more valuable to developers than a hallucinated fix.

---

🚀 What's Next for PatchPilot

• Multi-Language Expansion: Expanding beyond Python to full containerized sandbox runners for TypeScript/JavaScript (vitest, jest), Go, and Rust.
• GitHub App & PR Bot: Automatically trigger PatchPilot when a CI workflow fails, posting verified unified diffs directly to the Pull Request.
• Interactive Patch Editor: Allow developers to adjust the unified diff directly in the browser before triggering the verification stage.
• Automated Regression Test Synthesis: Have Nemotron not only fix the bug, but also generate new edge-case tests to prevent future regressions.

Built With

• python
• typescript