MergeReady

Check your first pull request before a maintainer has to. MergeReady reads a repo's own rules and shows what could block a first-time contributor, with proof for every rule. No AI inside.

Comment

Inspiration

New contributors often get their first pull request sent back for reasons that have nothing to do with their code: a missing sign-off, an unticked checklist, a commit message in the wrong format, or an issue someone else was already working on. On one "good first issue" in stdlib, two newcomers each opened a pull request for the same issue, and nobody was assigned. Big projects use bots to catch these problems, but bots only speak up after the pull request is public. I wanted something that helps before that.

What it does

Paste a link to a GitHub issue or pull request. MergeReady reads that repository's own contribution rules and checks it.

  • Issue link: is it assigned, already claimed in the comments, or already being fixed by open pull requests?
  • Pull request link: commit or PR-title format, DCO sign-off, PR template sections and checkboxes, linked issue, assignment, and tests changed when code changed.

Results appear as a journey map (Issue, Work, Commit, PR, Merge). Each check explains the problem in plain English, gives copyable fix commands, and shows its evidence: the exact sentence, config file or commit history the rule came from. It also drafts a PR description from the repo's template, but never ticks checkboxes for you. Only you can make those promises.

How I built it

Next.js 16, TypeScript, Tailwind CSS and the GitHub REST and GraphQL APIs, deployed on Vercel, with 430 tests.

There is no AI inside the product. Rules are extracted with deterministic code from four sources: config files, the PR template, the repo's commit history, and sentences in the contributing docs. The same input always gives the same answer, and every red or yellow flag can be checked. A failed rule is red only when the source clearly requires it; otherwise it is yellow.

I built it with AI assistants (Claude Code and Claude). Every step is disclosed in the README and AI_LOG.md.

Challenges I ran into

  • Passing tests, broken reality. My mocked tests all passed, but on real data my fine-grained GitHub token was hiding the cross-referenced events the tool needs to find open pull requests. A classic token with no scopes fixed it.
  • Messy real repositories. React's contributing guide is one link, and Node's points to nine other documents. Some rules are never written in a readable form at all, so I taught MergeReady to learn them from commit history.
  • Too many false alarms. My first evaluation on 80 real pull requests showed only 10% of flags were correct, mostly because example lines, HTML comments and "If it applies" lines in templates were treated as required.

Accomplishments that I'm proud of

  • Every rule comes with proof, and the tool never guesses a git command.
  • An honest, pre-registered evaluation: selection rules committed before running, pull requests rebuilt as they were first submitted, and labels locked before scoring.
Run Recall Precision
First version, 80 PRs 9/18 9/90
After fixes, same 80 PRs (optimistic) 9/18 9/31
Held-out: 19 new PRs (main result) 3/5 3/7

The held-out sample is small, so the uncertainty is large. Full method: eval/REPORT.md in the repo.

What I learned

Passing tests is not the same as working, so test on real data early. Measure yourself honestly, and use a held-out test so you don't fool yourself. Check that an idea doesn't already exist before building it. And directing AI well means asking for plans, questioning every result, and rejecting changes you don't understand.

What's next

Testing with more first-time contributors, a larger evaluation, checking commit line length and subsystem names, and following more linked documents.

Built With

Share this project:

Updates

Submission history