Inspiration

Autonomous AI agents are transforming how individuals and businesses operate daily, yet interacting with payment ecosystems remains highly fragmented. Traditional payment gateways require multi-step manual forms, repetitive data inputs, and separate manual verification interfaces. We envisioned AgentPay AI to seamlessly bridge conversational intelligence with real-world financial rails, enabling users to securely execute end-to-end payments through intuitive, natural language commands.

What it does

AgentPay AI is an autonomous, privacy-focused payment co-pilot that simplifies transaction lifecycles into a frictionless workflow:

  • Natural Language Understanding: Interprets conversational user commands (e.g., "Send $40 to Alex for cloud server") and structures them into strict financial schemas.
  • PayPal Sandbox Order Creation: Automatically interfaces with PayPal REST v2 endpoints to construct valid transaction payloads and generate active Order IDs (PAYID-XXXXX).
  • Autonomous 2FA OTP Extraction: Incorporates a dedicated regex-driven security engine that intercepts incoming 2FA verification SMS payloads and autonomously extracts the 6-digit one-time passcode.
  • Automated Settlement: Verifies the extracted token and completes payment capture with real-time settlement telemetry.

How we built it

  • Backend: Built with FastAPI (Python) for high-performance, asynchronous REST API endpoints and CORS handling.
  • Local AI Engine: Integrated a locally hosted Mistral model using Ollama via HTTP REST streaming to ensure complete financial data privacy, zero external API costs, and sub-second inference.
  • Payment Sandbox: Modeled around PayPal REST v2 specifications (/v2/checkout/orders and capture flows) to demonstrate compliant settlement.
  • Security & Extraction: Engineered an isolated regex pattern extraction module for parsing one-time verification tokens from raw text feeds.
  • Frontend Dashboard: Designed a responsive, multi-stage telemetry dashboard using modern HTML5, JavaScript (Fetch API), and Tailwind CSS.

Challenges we ran into

  • Strict Schema Enforcement: Ensuring local LLMs consistently produce structured JSON without conversational preambles or formatting hallucinations. We engineered robust system prompts with JSON-only schemas and regex fallbacks.
  • Simulated 2FA Synchronization: Creating an authentic lifecycle where asynchronous security tokens are captured and validated against pending transaction identifiers without blocking the primary request loop.

Accomplishments that we're proud of

  • Successfully executing an end-to-end payment workflow—from raw text to settled PayPal order—in under two seconds.
  • Implementing an architecture where no sensitive payment intent data leaves the local machine during AI processing.
  • Creating an intuitive, transparent UI that visualizes every transition state of the autonomous agent in real time.

What we learned

  • How to reliably bridge local LLM inference engines with transactional backend architectures.
  • Best practices for designing automated two-factor verification pipelines while keeping users fully informed via telemetry.
  • Techniques for building lightweight, reactive dashboards that communicate smoothly with asynchronous Python services.

What's next for AgentPay AI

  • Real-time webhook integration with production SMS providers (Twilio) for live hardware 2FA interception.
  • Dynamic multi-currency conversion and international cross-border payment routing.
  • Configurable biometric approval thresholds and spending allowances for fully unattended autonomous agent tasks.

Built With

Share this project:

Updates

Submission history