SENTINEL

SENTINEL transforms suspicious PayPal payments into explainable decisions with AI-powered fraud detection, evidence-backed investigations, and human-approved recovery workflows.

Comment

Inspiration

Every digital payment tells a story. But when a suspicious transaction is detected, a fraud score alone isn't enough to tell that story.

Merchants and risk analysts still need to understand why a payment was flagged, investigate behavioral patterns, gather supporting evidence, determine possible financial exposure, and decide what action to take.

Traditional fraud workflows can be fragmented across transaction dashboards, risk engines, case-management systems, and manual recovery processes. This makes investigations slower, less transparent, and harder to audit.

We started with one question:

What if fraud detection didn't end with an alert, but initiated an intelligent, explainable, and human-controlled investigation?

That idea became SENTINEL — AI Payment Trust & Recovery.

Our vision was to build more than a fraud classifier: a unified platform that connects PayPal payments, explainable machine learning, agentic investigations, evidence management, and accountable human decisions.

Follow the money. Explain the risk. Keep humans in control.

What it does

SENTINEL is an end-to-end payment-risk intelligence and investigation platform that helps merchants move from suspicious transactions to evidence-backed decisions.

The platform brings together several interconnected capabilities:

1. Explainable fraud detection

SENTINEL uses an XGBoost machine learning model trained on 12 historical behavioral features to assess transaction risk. Payments receive calibrated scores and LOW, MEDIUM, or HIGH classifications.

TreeSHAP explanations reveal which transaction and payer-history signals influenced each prediction, helping analysts understand why a payment deserves attention.

2. Gemini-powered AI investigations

SENTINEL integrates Google Gemini 3.1 Flash Lite through a constrained investigation agent.

Rather than generating unsupported conclusions, the agent retrieves authorized case records through read-only tools, examines transaction details and payer history, identifies risk and counter-indicators, and produces structured findings with source references.

A verified hosted investigation demonstrated seven recorded tool calls and evidence-grounded findings.

3. Evidence and recovery intelligence

Analysts can manage private evidence, identify missing documents, generate PDF investigation packets, prepare dispute responses, and review refund-related requests.

AI provides recommendations, while sensitive financial decisions remain subject to human authorization and approval controls.

4. Intelligent fraud operations

The Command Center and transaction explorer provide centralized views of payment activity, risk classifications, investigation queues, transaction histories, and review priorities.

5. Policy Lab and monitoring

Merchants can simulate changes to fraud thresholds and investigate their effect on precision, recall, false positives, operational capacity, and estimated financial costs.

Risk Monitoring helps visualize score distributions, payment trends, and operational behavior.

6. Relationship Explorer

An interactive network connects related payer references, payments, and investigation cases, helping analysts explore suspicious relationships through recorded evidence.

7. PayPal and event-processing integrations

SENTINEL integrates PayPal Sandbox payment workflows with Supabase-backed webhook ingestion, durable event processing, reconciliation, and automated scoring.

The platform explicitly distinguishes Sandbox transactions from simulated and synthetic demonstration records.

The result is a connected workflow:

Payment → Risk Detection → Explanation → AI Investigation → Evidence → Human Approval → Recovery Preparation.

How we built it

SENTINEL was built as a modular full-stack financial technology application, combining machine learning, agentic AI, payment infrastructure, and modern web engineering.

Frontend

We developed the interface using React, TypeScript, Vite, Tailwind CSS, and GSAP, creating an interactive financial operations workspace.

The application includes a cinematic "Follow the Money" landing experience, an accessible read-only judge demo, interactive transaction dashboards, investigation workbenches, policy simulations, and relationship visualizations.

Backend and infrastructure

We used Python, FastAPI, PostgreSQL, SQLAlchemy, and Alembic to implement payment processing, risk assessment, investigation management, evidence storage, access controls, and audit logging.

The application is deployed to Render, with Supabase supporting webhook ingestion and durable event processing.

Machine learning

We developed a reproducible XGBoost fraud-risk pipeline using scikit-learn, NumPy, and SHAP.

The synthetic benchmark contains:

  • 9,051 transactions across 3,000 generated payers
  • 12 past-only behavioral features
  • Chronological 60/20/20 training, validation, and test splits
  • Comparisons against rules-based and logistic regression baselines
  • Platt probability calibration and TreeSHAP explainability

Our selected model achieved the following results on the held-out synthetic test set:

Metric Result
ROC-AUC 0.841
PR-AUC 0.348
Recall 61.5%
Precision 28.2%

These measurements describe reproducible synthetic testing, not production-validated fraud detection.

Agentic AI

We integrated Google Gemini through PydanticAI, using typed, read-only investigation tools.

The agent produces structured findings supported by authorized case records. Source validation, bounded tool access, and audit trails help prevent unsupported claims and unauthorized actions.

The application also supports an explicitly labeled deterministic investigation mode when an external AI provider is unavailable.

PayPal integration

We implemented PayPal Sandbox checkout, OAuth authentication, order and capture handling, webhook processing, and reconciliation.

A Supabase-backed event pipeline supports durable ingestion, retries, deduplication, and downstream scoring.

Hosted staging checks have confirmed PayPal Sandbox authentication and Gemini investigation functionality. Financial execution remains restricted to the project's authorized test and preparation workflows.

Testing and reliability

We used pytest, Playwright, GitHub Actions, Docker, and automated integration tests to validate application behavior, model reproducibility, security controls, and deployment readiness.

Challenges we ran into

Building realistic fraud detection without sensitive customer data

Access to real payment-fraud datasets is limited by privacy and availability. We developed a deterministic synthetic dataset with multiple legitimate and suspicious behavioral patterns, including difficult-to-detect activity and high-frequency legitimate payers.

Preventing misleading machine learning results

Fraud classification involves severe class imbalance. We evaluated precision-recall trade-offs, calibrated predictions, applied chronological splitting to prevent future-information leakage, and examined operational review capacity rather than relying on accuracy alone.

Making AI investigations trustworthy

A language model can generate convincing but unsupported explanations. We implemented read-only tools, source validation, structured findings, and strict authorization boundaries so investigations remain grounded in accessible records.

Handling real-world payment infrastructure

PayPal Sandbox authentication, webhook delivery, external API configuration, network restrictions, and asynchronous processing introduced significant integration challenges.

We developed retry-safe processing, idempotency controls, durable event handling, and separated simulation from verified Sandbox activity.

Balancing technical depth with usability

Combining transaction monitoring, model explainability, agentic investigations, evidence management, and policy simulations within one understandable interface required substantial frontend and workflow design.

Accomplishments that we're proud of

Building a complete investigation workflow rather than another standalone fraud detector. SENTINEL connects risk detection to explainability, investigation, evidence management, approvals, and recovery preparation.

Developing a reproducible XGBoost risk engine with 12 historical features, calibrated predictions, TreeSHAP explanations, and transparent held-out evaluation results.

Successfully demonstrating hosted Gemini investigation functionality, including seven recorded tool calls and evidence-grounded findings.

Integrating PayPal Sandbox and Supabase event-processing infrastructure, connecting payment activity to transaction reconciliation and risk-assessment workflows.

Implementing human-controlled financial workflows, including independent approvals, evidence packets, dispute preparation, and detailed audit trails.

Creating interactive decision intelligence, including the Policy Lab, risk monitoring, and relationship exploration.

Building an accessible public judge experience featuring a cinematic payment story and isolated, read-only synthetic demonstration.

Most importantly, we created a working foundation for a financial-risk platform where machine learning provides intelligence, AI helps interpret evidence, and humans remain responsible for consequential decisions.

What we learned

Building SENTINEL reinforced that fraud detection is not simply a machine learning classification problem. It is an end-to-end decision-making problem.

A risk score becomes more useful when investigators can understand its causes, examine supporting evidence, and determine an appropriate response.

We learned that model precision, recall, calibration, and operational review capacity must be considered together. A technically strong classifier can still create substantial operational costs if its alerts are poorly prioritized.

We also learned that trustworthy agentic AI requires more than an effective language model. It needs constrained permissions, verifiable sources, structured outputs, clear failure handling, and human oversight.

Integrating PayPal and Supabase demonstrated the importance of idempotency, asynchronous processing, durable storage, reconciliation, and distinguishing verified external events from simulations.

Above all, we learned that AI should strengthen human financial judgment, not silently replace it.

What's next for SENTINEL

Our next goal is to evolve SENTINEL from a hackathon-stage platform into a more extensively validated payment-risk intelligence system.

Real-world model evaluation: Evaluate performance on appropriately licensed fraud datasets and investigate generalization beyond synthetic payment patterns.

Advanced fraud networks: Expand relationship analysis using graph-based machine learning and more sophisticated behavioral clustering.

Enhanced AI investigation: Improve multi-step evidence gathering, investigation prioritization, model evaluation, and source-grounded recommendations.

Recovery workflow expansion: Extend dispute preparation, evidence automation, and approval-controlled financial operations with additional verification and safeguards.

Production readiness: Strengthen observability, scaling, deployment security, integration testing, and operational resilience.

Continuous policy intelligence: Develop improved threshold recommendations, model-drift detection, and review-capacity forecasting.

Our long-term vision is to create a financial intelligence platform that doesn't just answer:

"Is this transaction suspicious?"

But goes further:

"Why is it suspicious, what evidence supports that assessment, and what should we do next?"

Because detecting risk is only the beginning.

SENTINEL — Follow the money. Explain the risk. Keep humans in control.

Built With

Share this project:

Updates

Submission history