ReproBot

Inspiration

Every maintainer's least favorite notification: a new issue that says "it crashes sometimes." No version, no snippet, no environment. Triage is the silent tax on open source — reproducing reported bugs is the biggest time sink in OSS maintenance. LLM tools that summarize issues don't help, because the hard question isn't "what does the report say?" — it's "is the bug real, and what's the smallest script that proves it?"

What it does

ReproBot closes the loop that other tools leave open: it executes what it writes.

1. Paste a GitHub issue URL (plus an optional git ref to pin the reported version).
2. A GPT-5.6 agent fetches the issue and comments, shallow-clones the repo, and explores the source with tool calls.
3. It writes a minimal reproduction script — then runs it in a sandboxed subprocess with timeouts.
4. It compares the observed output against the reported failure. Not matching? It iterates — reading more code, rewriting the script — up to six rounds.
5. You get a live streaming agent log and a final verdict: Reproduced / Not reproduced / Needs info, the repro script, the actual captured output, a suggested fix, and one-click "copy as issue comment."

Works on Python and JavaScript/TypeScript repositories. We demo it on real historical bugs from more-itertools, python-dateutil, and lodash — all verified by execution, not guesswork.

How we built it (with Codex and GPT-5.6)

The core of ReproBot — the agent loop in lib/agent.ts, the execution sandbox in lib/sandbox.ts, the GitHub ingestion, the SSE streaming API, and the UI — was built in a single Codex session (Session ID submitted with this entry). Our workflow: we wrote a detailed product spec, handed it to Codex as the master prompt, reviewed every diff, and steered with follow-up prompts in the same thread. Codex generated the tool-calling architecture, the subprocess sandboxing with allowlisted commands, and the streaming UI; we made the product decisions (verification-first design, pinned refs for deterministic reproduction, allowlist-only command execution) and tested against real issues.

GPT-5.6 appears twice: it's the model inside the product (driving the reproduction agent via function calling), and it powered Codex while building the product. ReproBot can run its agent on the Codex CLI (free with a ChatGPT plan — no API credits needed) or the OpenAI API. The README documents the full collaboration.

Challenges we ran into

• Verification semantics: "the script failed" is not "the bug reproduced." The agent must compare how it failed against the report — a wrong return value is a reproduction even when the exit code is 0 (our primary demo bug silently returns wrong data!).
• Safe execution: running unknown repo code means strict allowlists (only pip install/npm install for setup, only repro.* scripts for execution), timeouts, temp-dir isolation, and output caps.
• Old code, new interpreters: some historical bugs can't even import on modern Python — the agent has to distinguish "environment failure" from "bug not present" and report honestly.

Accomplishments that we're proud of

A demo where every result is proven by execution — and an honest verdict system that says "not reproduced" rather than hallucinating success.

What's next for ReproBot

GitHub App that triages new issues automatically as they're filed; container-level isolation; auto-PR of the suggested fix with the repro as a regression test.

Built With

• codex
• gpt-5.6
• next.js
• node.js
• openai
• typescript