PR Impact Analyzer

A GitHub PR reviewer that maps dependency risk before merge and turns the graph into a clear IBM Bob report.

Comment

Inspiration

Most pull request tools explain what changed, but reviewers still have to guess what the change might break. That gets risky when a small edit touches shared code, import paths, tests, or downstream modules.

I built PR Impact Analyzer for AlgoFest because the core problem is algorithmic: start from the changed file, follow the dependency graph, score the blast radius, and give the reviewer evidence before merge.

What it does

PR Impact Analyzer listens for GitHub pull request events through a GitHub App webhook. When a PR opens, it fetches the PR diff, repository tree, and selected source context.

The app builds a weighted dependency graph from imports and relative module references. It starts from the changed files, walks reverse dependency paths, and calculates affected files, critical paths, likely test risk, and a 0-100 merge risk score.

IBM Bob receives the graph evidence and turns it into a readable engineering report. The result is posted back to the pull request as a GitHub comment and shown in the dashboard, so reviewers can see the risk, affected files, and tests to run before merge.

How I built it

The backend is a FastAPI app with routes for the dashboard, health checks, manual analysis, API access, and GitHub webhooks. GitHub App authentication uses PyGithub, with webhook signature validation before analysis runs.

The graph analyzer parses repository files into nodes and import relationships into weighted edges. It uses changed PR files as start nodes, performs bounded graph traversal, identifies affected dependents, and scores risk from affected node count, edge weight, change size, critical path depth, and test coverage gap.

For the AI layer, the app supports IBM Bob through an OpenAI-compatible chat completions endpoint using the granite-8b-code-instruct model. Bob does not guess the risk from scratch. It receives the graph output and writes a structured impact summary that the app parses into the PR comment and dashboard.

Challenges

The hardest part was keeping the analysis grounded. A plain LLM summary of a diff can sound useful while missing the actual blast radius. I separated the system into two layers: deterministic graph analysis first, Bob explanation second.

I also wanted the proof to be real. The demo uses a live GitHub PR, a webhook path, Bob-backed analysis, a GitHub PR comment, dashboard evidence, and passing tests.

Accomplishments

I shipped a working GitHub App flow that can receive a PR event, inspect the repository, calculate graph-backed impact, ask IBM Bob for a structured explanation, post the report back to GitHub, and preserve the report in a dashboard.

The output is explainable. Reviewers can see why a PR is risky: affected nodes, weighted edges, critical path, coverage gap, and suggested tests are all exposed instead of hidden behind a black-box answer.

What I learned

AI is most useful in code review when it explains evidence instead of replacing it. The graph traversal gives the system something concrete to reason from, and Bob makes that output readable enough for a reviewer to act on quickly.

What's next

Next I would add persistent database storage, support for more languages, richer test mapping, and a stronger dashboard for comparing risk across PRs over time. I would also move analysis into an async job queue so larger repositories can be processed without blocking webhook responses.

Built With

Share this project:

Updates