RepoMedic

Inspiration

When a production regression fires at 3:00 AM, on-call engineers spend 45 to 90 minutes manually jumping between alert stack traces, Git commit histories, local test runners, documentation tabs, and pull request templates. Yet standard AI coding assistants fail in high-stakes incident response because they guess without executing—emitting speculative code snippets in a chat window without proving the failure was reproduced, without verifying the full test suite passes in a clean room, and without checking whether tests were weakened or security regressions introduced.

We built RepoMedic — Diagnose. Patch. Prove. to turn production exceptions into test-backed, security-audited, cryptographically attested GitHub Draft Pull Requests, where every reasoning step, sandbox exit code, and diff hash is transparent and verifiable.

What it does

RepoMedic is an autonomous closed-loop production incident investigator and self-verifying coding agent:

1. Deterministic Bug Reproduction (exit 1): Pins the exact Git commit SHA of an allowlisted repository, provisions an isolated clean-room sandbox with zero application secrets, and runs the test suite to confirm the failure is genuinely reproduced before touching a single line of code.
2. Grounded Root-Cause Diagnosis: Calls nvidia/nemotron-3-super-120b-a12b on Nebius Token Factory, enriched with cross-incident Neural Memory (prior verified fixes) and live Tavily Search API documentation grounding, producing a strict Zod-validated DiagnosisSchema with line-linked file evidence and confidence scores.
3. Surgical Patch Synthesis & Bounded Reflection: Proposes a minimal unified diff touching only the root-cause lines—never altering unit tests or lockfiles—with bounded self-healing reflection (Attempt 1..2) if candidate sandbox verification fails.
4. Dual Clean-Room Sandbox Proof (exit 1 -> exit 0):
   • Vercel Sandbox (@vercel/sandbox): Runs pnpm install --ignore-scripts and npx vitest run in ephemeral Node 22 microVMs to prove all tests transition from failing to green.
   • Nebius Token Factory Sandboxes (contree-sdk Beta): Spawns 3-layer Copy-on-Write (CoW) Linux containers from python:3.12-slim, dynamically installs qrencode via apt-get, verifies Python AST/boundary invariants on the patch, snapshots the immutable CoW layer UUID, and renders an ANSI UTF-8 QR verification badge.
5. 6-Gate Static Security & Blast-Radius Audit: Audits every candidate diff across 6 deterministic security rules (NO_DYNAMIC_CODE_EXEC, NO_SHELL_INJECTION, NO_SECRET_LITERALS, NO_TEST_TAMPERING, NO_DEPENDENCY_DRIFT, CRYPTOGRAPHIC_DIFF_INTEGRITY) and assigns a 0–100 Surgical Precision Score.
6. LangSmith 6-Span Observability & SLSA-256 Attestation: Streams a 6-span hierarchical agent trace tree plus 4 automated online evaluator scores (sandbox_verification_pass, security_blast_radius_score, nemotron_diagnosis_confidence, mttr_compression_ratio) to LangSmith, exports downloadable SLSA-256 Attestation JSON, and gates GitHub Draft PR creation behind explicit human approval of the 64-character SHA-256 diff hash.

How we built it

• AI Inference & Reasoning: Direct OpenAI-compatible integration with Nebius Token Factory (https://api.tokenfactory.us-central1.nebius.com/v1/) running NVIDIA Nemotron (nvidia/nemotron-3-super-120b-a12b) for root-cause diagnosis, unified diff synthesis, bounded reflection retries, and an interactive Incident Copilot.
• Dual Isolated Sandboxes: Combined Nebius Token Factory Sandboxes (Beta) (contree-sdk Copy-on-Write Linux VM sessions under project aiproject-e00am2xdmcyzgvf9s5) with Vercel Sandbox (@vercel/sandbox ephemeral Node 22 microVMs) so untrusted repository tests never execute inside web server functions or touch app credentials.
• Live Documentation Grounding (Tavily): Integrated Tavily Search API (src/lib/ai/tavily.ts) to ground Nemotron's root-cause analysis in authoritative language/SDK specifications (for example, distinguishing ECMAScript nullish coalescing ?? vs. logical OR || falsy coercion on 0% tax-exempt B2B orders).
• Agent Observability (LangSmith): Built automatic LLM call tracing and a 6-span hierarchical run-tree publisher (src/lib/observability/langsmith.ts) with 4 online evaluators and 1-click public shareable trace URLs.
• State Machine, Auth & Multi-Tenant Security: Built on Next.js 15 (App Router), React 19, TypeScript, and Supabase PostgreSQL 15 across 9 normalized tables protected by FORCE ROW LEVEL SECURITY, PostgreSQL RBAC triggers (admin_set_user_role, guard_profile_role_update), atomic stage leases (claim_run_stage), Octokit pinned-SHA GitHub integration, and Resend transactional emails.

Challenges we ran into

1. Reasoning-Model Structured Output Reliability: Large reasoning models like nemotron-3-super-120b-a12b emit deep chain-of-thought reasoning alongside structured JSON. We built a resilient parser (src/lib/ai/nebius.ts) that extracts structured payloads cleanly while preserving reasoning telemetry and enforcing strict Zod schemas before any state transition.
2. Serverless Execution Timeouts vs. Multi-Stage Verification: Cloning pinned commits, running baseline sandbox suites, calling 120B inference, and re-running patched suites can exceed single serverless request budgets if run monolithically. We solved this with an atomic PostgreSQL-leased step-by-step state machine (claim_run_stage) with automatic stale-lease recovery (>120s).
3. Copy-on-Write Sandbox Operation Polling: Integrating Nebius Token Factory Sandboxes (contree-sdk) across both Python and serverless TypeScript required handling asynchronous operation states (ASSIGNED -> PENDING -> EXECUTING -> SUCCESS) across chained CoW image layers (apt-get update -> apt-get install qrencode -> Python invariant runner).
4. Preventing Subtle AI Test Tampering: A coding agent could trivially make a red test suite green by deleting assertions or adding .skip. We engineered a deterministic 6-gate patch scanner (src/lib/security/patch-scanner.ts) that rejects any candidate patch touching test files, lockfiles, workflows, or dynamic execution primitives.

Accomplishments that we're proud of

• 100% Live End-to-End Reproducibility: Achieved 3/3 consecutive verified live closed-loop runs against a real GitHub fixture repository (ensuredesign/repomedic-fixture@440255c3e1ef), compressing incident triage from ~45 minutes to ~42 seconds with 100/100 Surgical Precision Scores (exit 1 -> exit 0).
• Defense-in-Depth Trust Architecture: Zero unverified patches, zero secrets exposed to sandboxes, FORCE ROW LEVEL SECURITY across all 9 Postgres tables, cryptographic SLSA-256 provenance export, and SHA-256 diff-hash human approval before opening GitHub Draft PRs.
• Judge-First Experience: Shipped 1-click Developer/Judge and Platform Admin logins (/auth), a zero-auth interactive Case Study (/demo), live Nebius ConTree QR attestation, and 1-click public LangSmith 6-span traces.

What we learned

• Execution Proof Beats Prompting Alone: Pairing NVIDIA Nemotron 3 Super 120B with deterministic clean-room sandboxes and bounded reflection transforms an LLM from a speculative code assistant into a trustworthy software engineering teammate.
• Copy-on-Write Sandboxes Unlock Instant Toolchains: Nebius Token Factory's contree-sdk CoW image layering makes it practical to install system packages (apt-get) once per layer and branch deterministic verification environments in seconds.
• Grounded Context Reduces Patch Blast Radius: Injecting targeted Tavily documentation snippets and historical Neural Memory before patch synthesis eliminated multi-file over-editing and produced clean 1-line surgical diffs.

What's next for RepoMedic

• Webhook-Driven CI & Sentry/Datadog Ingestion: Automatically trigger RepoMedic investigations when GitHub Actions fail or production error trackers detect a new regression spike.
• Pre-Warmed Nebius ConTree Language Images: Cache pre-built CoW base layers for Python, Go, Rust, and monorepo Node toolchains to cut sandbox cold-start latency under 3 seconds.
• Multi-File AST Call-Graph Slicing: Expand repository context retrieval to trace cross-module TypeScript/Python call graphs across 100k+ LOC enterprise monorepos.

Built With

• contree-sdk
• github-api
• langsmith
• nebius-sandboxes
• nebius-token-factory
• nextjs
• nvidia-nemotron
• octokit
• playwright
• postgresql
• python
• react
• resend
• supabase
• tailwindcss
• tavily
• typescript
• vercel
• vercel-sandbox
• vitest
• zod