Everything in this project runs on the PayPal Sandbox (test money).

Inspiration

Open-source maintainers and small startups want to pay outside developers for fixing bugs, but paying them is slow and risky. Someone has to review the pull request, watch for spam or malicious code, send an invoice, and move the money by hand. Contributors worry they'll do the work and never get paid. I wanted to see what happens when an AI agent sits between the two and enforces clear rules, so paying for good work takes no manual steps and bad work never gets paid.

What it does

PR-BountyPay is an autonomous escrow agent for GitHub bounties.

  1. Lock: A maintainer locks a bounty on a GitHub issue. PayPal places an authorization hold, so the funds are reserved but not yet moved.
  2. Claim: A contributor opens a pull request with Fixes #N and paypal-bounty: <their PayPal email> in the description. The first PR to claim an issue wins.
  3. Test: GitHub Actions runs the unit tests. A PR that edits the tests or the workflow fails automatically.
  4. Audit: If the tests pass, the agent reads the diff. It runs a fixed pattern scan for hard-coded secrets, then asks Gemini for a structured security review. Gemini is also instructed to flag any text in the code that tries to manipulate the reviewer.
  5. Pay: Only if tests pass, no secrets are found and the score is 95 or higher, the agent captures the PayPal hold and sends a PayPal Payout to the contributor.
  6. Hold: Otherwise nothing moves, and the live ledger shows exactly why: CI failed, secret leak, or low score.

I tested three outcomes in the PayPal sandbox:

  • Paid: a login-check fix scored 100/100 and the contributor received $120. Gemini had earlier blocked my first version at 90/100 because it used a non-constant-time password comparison. After I switched to hmac.compare_digest, it was paid.
  • Blocked: a PR that added a hard-coded credential scored 0/100, and the funds stayed locked.
  • Failed: a PR that deleted a test failed CI, so no audit ran and no money moved.

How I built it

  • Backend: Python and FastAPI. It receives signed GitHub webhooks (pull_request and workflow_run), keeps a SQLite ledger with a strict state machine, and runs the audit and payout in the background.
  • PayPal: Orders v2 with intent=AUTHORIZE for the escrow hold, Payments v2 to capture it, and the Payouts API to pay the contributor.
  • AI: Google Gemini through the google-genai SDK, with a typed JSON schema for the result (score, secrets found, injection attempt, one-line verdict). Temporary errors are retried, and an optional fallback model can take over.
  • CI: A GitHub Actions workflow that runs pytest and blocks edits to tests/ and .github/.
  • Dashboard: React, Vite, Tailwind and AG Grid. It shows live status pills, CI results, audit scores, PayPal IDs and a per-bounty inspection drawer, and it lets a maintainer lock new bounties through a protected API.

Safeguards

Because the contributor controls the code being judged, the AI is never the only gate:

  • GitHub webhook signatures (HMAC) are verified, and the write endpoints need an API key.
  • The audit is pinned to the commit CI tested. If newer commits arrive, the audit is skipped and the newer run decides.
  • The first PR to claim an issue wins, and the payout email is locked after the first CI result, so editing the PR description can't redirect the money.
  • Hard-coded secrets are caught by a fixed pattern scan, not only by the model, and the diff is treated as untrusted text.
  • Oversized diffs are held instead of truncated, and a failed audit never leads to a payout.
  • The ledger is re-read right before paying. Payments use ledger guards plus deterministic PayPal request and batch IDs, so a redelivered webhook can't pay twice.

Challenges I ran into

  • PayPal account setup: Payouts failed with PAYOUT_NOT_AVAILABLE because my first sandbox business account was registered in India, and PayPal would not send payouts from it. Rebuilding the sandbox with US accounts fixed it.
  • Race conditions: a contributor can push a new commit while an old CI run is still finishing, which led to the commit-pinning and re-check logic.
  • Reliability of the model: model names changed and some requests returned 503 and 504 errors, so the audit retries and never pays on a failed audit.
  • Safe AI review: the diff is untrusted data and can contain instructions aimed at the reviewer, so it is isolated in the prompt and backed by hard checks.

What I learned

Letting an agent control money works only if every decision is checked by code that can't be argued with. The AI adds judgment on top of the tests, the secret scan and the tamper checks, but it doesn't replace them.

How I used AI

In the product: Google Gemini reviews every PR that passes CI. It looks for insecure patterns, authentication bypasses and manipulation attempts, and returns a structured verdict (score, secrets found, injection attempt, one-line summary) that decides whether the payout fires. A separate pattern scan checks for hard-coded secrets.

While building: I used an AI coding assistant (Claude) to plan the architecture, review my code and debug.

What's next

  • Live PayPal. This project is sandbox only, with test money.
  • Payout reconciliation: today PAID means PayPal accepted the payout batch (in the sandbox it completes within seconds). A production version should confirm each payout item through PayPal's payout status and webhooks, and recover safely from ambiguous timeouts.
  • Re-authorizing holds, since PayPal authorizations expire after days, which limits long-running bounties.
  • Stricter protection for renamed or nested test files, a static analyzer (such as Bandit or Semgrep) in CI, and a ledger API with authentication and pagination.
  • Pull requests from forks, split payouts between several contributors, and automatic merging.

Built With

  • ag-grid
  • claude
  • fastapi
  • gemini-api
  • github-actions
  • github-api
  • github-webhooks
  • google-gemini
  • javascript
  • ngrok
  • paypal
  • paypal-payouts-api
  • pydantic
  • pytest
  • python
  • react
  • sqlite
  • tailwindcss
  • vite
Share this project:

Updates

Submission history