GhostRing – Evidence-Based Fraud Detection for Small Sellers
Inspiration
Most small sellers meet fraud the hard way: a chargeback arrives, a sale is lost, and a dispute can't be won. The tools built to stop it assume thousands of labelled cases. A small seller has a handful, and what those tools give back is a bare risk score — a number the seller can't act on, can't explain to a customer, and can't defend.
We built GhostRing for that seller. It starts from the frauds they already trust, puts the proof behind every flag in plain view, and never moves money on a guess.
What it does
GhostRing turns a PayPal seller's incoming payments into a ranked review queue. Each payment is compared with the seller's confirmed frauds and known-good payments, and every flag arrives with the three closest confirmed frauds attached as evidence. The seller never has to take a ranking on faith.
Responsibility is divided on purpose. Gemini drafts an explanation and proposes an action: refund, review, or approve. Our verifier checks the text and makes the final decision. A person confirms. PayPal then confirms that the payment can be refunded. No single step can move money alone.
How we built it
PayPal Integration
PayPal sits at the center of every step that touches money. The demo payments are real sandbox orders, created and captured through the Orders API.
When a person confirms a refund, the app reads the capture back from PayPal and refuses to continue unless it reports COMPLETED with a matching amount. Only then does it call the refund endpoint, under a PayPal-Request-Id idempotency key, so a repeated click cannot refund the same payment twice.
We call the REST API directly rather than through an agent toolkit, which keeps the step that moves money deterministic and auditable. Every call targets the sandbox, and the client refuses to start against any other host.
Graph-Based Fraud Detection
The scoring is our own work. GraphGhost spreads the seller's confirmed labels across a similarity graph of payments using label spreading on a 10-nearest-neighbour graph.
Each new payment inherits a score from the confirmed cases it most resembles. Nothing is trained. Built on scikit-learn and SciPy, it produces a 0–100 risk index that ranks payments against one another.
The underlying graph can be represented as:
$$G=(V,E)$$
where each vertex represents a transaction and edges connect transactions with similar feature representations.
Gemini and Deterministic Verification
Gemini sees only the facts the scorer produced, and we treat what it writes as untrusted input.
An explanation is accepted only if every number and transaction ID in it appears in those facts; otherwise, a template takes its place.
Code then fixes the final action: a proposed refund drops to review unless the risk index is at least 60 and the nearest confirmed fraud has a similarity of at least 0.70.
The policy can be expressed as:
$$\text{Refund} \iff (R \geq 60) \land (S_{\text{nearest}} \geq 0.70)$$
where:
- $R$ is the risk index.
- $S_{\text{nearest}}$ is the similarity score of the nearest confirmed fraud.
With no evidence, there is no refund. Four tests cover this behaviour.
Application Architecture
A FastAPI backend and a plain HTML and JavaScript frontend bring everything together:
- Transaction review queue.
- Graph-based fraud evidence.
- Gemini-generated explanations.
- Deterministic verifier verdict.
- PayPal payment records.
- Refund receipts and transaction status.
Built During the Hackathon
Before the Submission Period, there was only a research prototype of kNN-graph label propagation, tested on small textbook datasets (Iris and Wine).
Everything specific to this project was built for the event: fraud scoring and evidence at scale, the baseline comparison, the Gemini explainer and its merchant policy, the verifier and its tests, the PayPal sandbox integration, the web app, and the deployment setup.
Try It
Every route is free and needs no login.
Demo Video:
https://www.youtube.com/watch?v=xzbtp27wsbg
To run it with no keys, set LLM_MODE=mock and start the app. This provides scoring, evidence, the verifier, and the review queue on the bundled sample.
For live sandbox refunds, follow SETUP.md; test credentials for judges are in the testing instructions, not in the repository.
The four seeded payments have already been refunded and show as Refunded, so seed fresh ones to run a new refund.
Website Preview:
https://garlapatiraviteja.github.io/Graph-Ring/?demo
It shows the frontend only, with clearly labelled simulated data and no PayPal calls.
Challenges We Ran Into
A Fluent Model Is Not a Trustworthy One
Early on, the risk was not that Gemini would write badly but that it would write well and be wrong. A confident explanation can quote an amount or a transaction ID that the scorer never produced, and a seller has no way to spot that.
We decided the model would never get the last word. The verifier checks every number and ID against the scorer's facts, swaps in a template when anything fails, and lets code, not the model, set the final action.
It also fails closed: with no similarity evidence, nothing is refunded. This took more design effort than the explanation itself, and it is the part we trust most.
Holding Ourselves to an Honest Benchmark
Our first numbers looked better than they deserved to. The benchmark sample is enriched, at about 1.6% fraud against roughly 0.17% in the source data, which flatters PR-AUC.
Our method also scores a whole batch at once, while the baselines see only their labels.
We could have left both facts out. Instead, we put them in the README, reported that Random Forest beats us by about two points, and rested our claim on what is true: comparable accuracy with no training, plus evidence for every flag.
Making the Refund Step Strict
A flag only matters once it becomes money, so that step needed the most care.
We made PayPal the judge of whether a payment can be refunded, blocked any capture whose status or amount did not match, and added an idempotency key so a double-click could not pay out twice.
The sandbox-only client was a deliberate limit: it refuses to run against any other host.
Working with Data That Has No Identities
The public dataset is anonymised, with no accounts, devices, or emails. That meant the graph could link payments only by feature similarity, so we could not honestly claim to find fraud rings, and we do not.
A Step That Cannot Be Automated
PayPal requires a buyer to approve every order, so seeding the demo payments had to include one manual step.
We accepted that, wrote it into SETUP.md, and used it to prove the refunds against real sandbox captures.
Accomplishments That We're Proud Of
End-to-End PayPal Integration
Four sandbox payments were created, approved by a sandbox buyer, captured, and refunded through the app — $425.07 in test money in total.
PayPal shows each capture as REFUNDED.
| Transaction | Amount | PayPal Refund ID | Status |
|---|---|---|---|
| TX0135 | $77.89 | 8J376810CS489174J |
COMPLETED |
| TX0485 | $24.90 | 6LM96918AD5509200 |
COMPLETED |
| TX0081 | $83.38 | 8BK82312SS343845M |
COMPLETED |
| TX0700 | $238.90 | 0N516169PF7820533 |
COMPLETED |
Public Benchmark Evaluation
On the ULB credit-card dataset, GraphGhost reaches 0.81 to 0.83 PR-AUC with only 5 to 25 confirmed frauds and no training step.
That is within about two points of a Random Forest, which is slightly more accurate and much faster.
We claim comparable accuracy without training, not superiority.
| Known Frauds | GraphGhost | Logistic Regression | Random Forest |
|---|---|---|---|
| 5 | $0.811 \pm 0.019$ | $0.734 \pm 0.206$ | $0.832 \pm 0.015$ |
| 10 | $0.820 \pm 0.023$ | $0.744 \pm 0.065$ | $0.842 \pm 0.012$ |
| 25 | $0.827 \pm 0.026$ | $0.792 \pm 0.030$ | $0.853 \pm 0.014$ |
It is also far steadier than logistic regression when labels are scarce, with a standard deviation of 0.019 against 0.206 at five labels.
The figures are the mean over five seeds on 30,492 transactions, and the repository has the command to reproduce them.
Evidence-Backed Decisions
Every flag carries its proof, and a refund needs three things to agree:
- The code's approval.
- PayPal's confirmation.
- A person's click.
What We Learned
The biggest lesson was where to put trust. Letting the model propose while code decides, and letting PayPal confirm before money moves, made the whole system easier to reason about and to audit.
We also learned that a weaker result stated plainly persuades more than a stronger one overclaimed. And a risk score is
