-
-
Full dashboard : a green PAID 100/100 row, a red 0/100 SECRET LEAK row and a red CI FAILED row, with the stat cards visible
-
Architecture: GitHub webhooks and CI feed an agent that audits with Gemini and pays via PayPal.
-
A maintainer locks a bounty. PayPal reserves the funds with an authorization hold.
-
Three real PRs, three outcomes: paid, secret blocked, CI failed.
-
From [CI RESULT] ... PASSED through [GEMINI AUDIT] Score: 100/100, [RULE PASSED] and [PAYPAL SETTLED] with the capture and payout IDs
-
The developer’s PayPal sandbox page with the received $120.00 from Test Store in the activity list
-
Terminal for the blocked case: [GEMINI AUDIT] Score: 0/100, Secrets Found: True, [PAYOUT BLOCKED] ... Hold remains LOCKED
Everything in this project runs on the PayPal Sandbox (test money).
Inspiration
Open-source maintainers and small startups want to pay outside developers for fixing bugs, but paying them is slow and risky. Someone has to review the pull request, watch for spam or malicious code, send an invoice, and move the money by hand. Contributors worry they'll do the work and never get paid. I wanted to see what happens when an AI agent sits between the two and enforces clear rules, so paying for good work takes no manual steps and bad work never gets paid.
What it does
PR-BountyPay is an autonomous escrow agent for GitHub bounties.
- Lock: A maintainer locks a bounty on a GitHub issue. PayPal places an authorization hold, so the funds are reserved but not yet moved.
- Claim: A contributor opens a pull request with
Fixes #Nandpaypal-bounty: <their PayPal email>in the description. The first PR to claim an issue wins. - Test: GitHub Actions runs the unit tests. A PR that edits the tests or the workflow fails automatically.
- Audit: If the tests pass, the agent reads the diff. It runs a fixed pattern scan for hard-coded secrets, then asks Gemini for a structured security review. Gemini is also instructed to flag any text in the code that tries to manipulate the reviewer.
- Pay: Only if tests pass, no secrets are found and the score is 95 or higher, the agent captures the PayPal hold and sends a PayPal Payout to the contributor.
- Hold: Otherwise nothing moves, and the live ledger shows exactly why: CI failed, secret leak, or low score.
I tested three outcomes in the PayPal sandbox:
- Paid: a login-check fix scored 100/100 and the contributor received $120. Gemini had earlier blocked my first version at 90/100 because it used a non-constant-time password comparison. After I switched to
hmac.compare_digest, it was paid. - Blocked: a PR that added a hard-coded credential scored 0/100, and the funds stayed locked.
- Failed: a PR that deleted a test failed CI, so no audit ran and no money moved.
How I built it
- Backend: Python and FastAPI. It receives signed GitHub webhooks (
pull_requestandworkflow_run), keeps a SQLite ledger with a strict state machine, and runs the audit and payout in the background. - PayPal: Orders v2 with
intent=AUTHORIZEfor the escrow hold, Payments v2 to capture it, and the Payouts API to pay the contributor. - AI: Google Gemini through the
google-genaiSDK, with a typed JSON schema for the result (score, secrets found, injection attempt, one-line verdict). Temporary errors are retried, and an optional fallback model can take over. - CI: A GitHub Actions workflow that runs pytest and blocks edits to
tests/and.github/. - Dashboard: React, Vite, Tailwind and AG Grid. It shows live status pills, CI results, audit scores, PayPal IDs and a per-bounty inspection drawer, and it lets a maintainer lock new bounties through a protected API.
Safeguards
Because the contributor controls the code being judged, the AI is never the only gate:
- GitHub webhook signatures (HMAC) are verified, and the write endpoints need an API key.
- The audit is pinned to the commit CI tested. If newer commits arrive, the audit is skipped and the newer run decides.
- The first PR to claim an issue wins, and the payout email is locked after the first CI result, so editing the PR description can't redirect the money.
- Hard-coded secrets are caught by a fixed pattern scan, not only by the model, and the diff is treated as untrusted text.
- Oversized diffs are held instead of truncated, and a failed audit never leads to a payout.
- The ledger is re-read right before paying. Payments use ledger guards plus deterministic PayPal request and batch IDs, so a redelivered webhook can't pay twice.
Challenges I ran into
- PayPal account setup: Payouts failed with
PAYOUT_NOT_AVAILABLEbecause my first sandbox business account was registered in India, and PayPal would not send payouts from it. Rebuilding the sandbox with US accounts fixed it. - Race conditions: a contributor can push a new commit while an old CI run is still finishing, which led to the commit-pinning and re-check logic.
- Reliability of the model: model names changed and some requests returned 503 and 504 errors, so the audit retries and never pays on a failed audit.
- Safe AI review: the diff is untrusted data and can contain instructions aimed at the reviewer, so it is isolated in the prompt and backed by hard checks.
What I learned
Letting an agent control money works only if every decision is checked by code that can't be argued with. The AI adds judgment on top of the tests, the secret scan and the tamper checks, but it doesn't replace them.
How I used AI
In the product: Google Gemini reviews every PR that passes CI. It looks for insecure patterns, authentication bypasses and manipulation attempts, and returns a structured verdict (score, secrets found, injection attempt, one-line summary) that decides whether the payout fires. A separate pattern scan checks for hard-coded secrets.
While building: I used an AI coding assistant (Claude) to plan the architecture, review my code and debug.
What's next
- Live PayPal. This project is sandbox only, with test money.
- Payout reconciliation: today
PAIDmeans PayPal accepted the payout batch (in the sandbox it completes within seconds). A production version should confirm each payout item through PayPal's payout status and webhooks, and recover safely from ambiguous timeouts. - Re-authorizing holds, since PayPal authorizations expire after days, which limits long-running bounties.
- Stricter protection for renamed or nested test files, a static analyzer (such as Bandit or Semgrep) in CI, and a ledger API with authentication and pagination.
- Pull requests from forks, split payouts between several contributors, and automatic merging.
Built With
- ag-grid
- claude
- fastapi
- gemini-api
- github-actions
- github-api
- github-webhooks
- google-gemini
- javascript
- ngrok
- paypal
- paypal-payouts-api
- pydantic
- pytest
- python
- react
- sqlite
- tailwindcss
- vite