AgentGain

Inspiration

When PayPal announced its partnership with OpenAI to enable AI-powered shopping, it made us think about a future where AI agents wouldn't just perform tasks — they could also discover and purchase new capabilities.

But one question stood out: What if an AI agent pays for a skill that promises better performance, but doesn't actually deliver?

AI skills are reusable instructions and examples designed to improve an agent's abilities. Yet a skill that works well for one task may fail on another. How can buyers trust the seller's claims without testing the skill on their own tasks?

We also wondered: If a skill succeeds only partially, should the buyer pay the full price, or should the seller receive nothing?

These questions inspired AgentGain — a marketplace that connects AI skill evaluation directly to payment.

By combining AI-based testing with PayPal's authorization and capture capabilities, we wanted to create a system where buyers pay for demonstrated value, not just promised performance.

Our guiding belief: In the age of AI agents, proof should come before payment.

What it does

AgentGain is a test-before-you-pay marketplace for AI skills — reusable instructions and examples that improve an AI assistant's performance.

Buyers can choose Full Skill, paying for reusable access only after successful evaluation, or Verified Work, paying only for accepted outputs.

AgentGain compares AI performance with and without the skill, verifies the results, and uses PayPal to determine payment settlement.

Our OneIntent feature also identifies overlapping purchase requests, with AG Grid displaying the supporting evidence.

How we built it

We built AgentGain using FastAPI, Python, PostgreSQL/Neon, and a browser-based frontend.

Ollama with Qwen 2.5 evaluates AI performance with and without a skill.
Deterministic verification checks results against buyer-provided answers and evaluation rules.
PayPal Sandbox APIs handle authorization, capture, and void based on evaluation outcomes.
Elasticsearch and Kibana Agent Builder help identify overlapping purchase intentions.
AG Grid provides an interactive view of evaluation and purchase evidence.

We also implemented frozen evaluation contracts, payment-state tracking, and a background evaluation worker to connect the process.

Challenges we ran into

One question challenged our original idea: What if a skill works for some tasks but fails others? Should the buyer pay everything, or should the seller receive nothing?

We realized a skill couldn't simply be sold in half.

That led us to introduce two payment models: Full Skill, where the complete skill must pass evaluation, and Verified Work, where payment is calculated only for individually accepted outputs.

Another challenge was coordinating AI evaluation with PayPal's payment lifecycle. We had to distinguish authorization from actual payment capture and ensure settlement decisions came from verified backend results, not the browser.

Accomplishments that we're proud of

We're most proud of turning a simple idea into a working prototype connecting AI evaluation, payment authorization, conditional settlement, and evidence tracking.

Instead of integrating PayPal as just another checkout button, we made it part of the product's core logic: the evaluation outcome determines whether and how much payment is captured.

Bringing PayPal, Ollama, Elasticsearch, AG Grid, and our backend together into one experience was a major achievement for our team.

What we learned

We learned that building trustworthy AI commerce involves much more than connecting AI to a payment API.

An AI-generated answer isn't automatically correct, a payment authorization isn't a completed payment, and a successful purchase doesn't necessarily mean the buyer received value.

The biggest lesson was that AI can perform the work, but clear verification rules should determine what deserves payment.

What's next for AgentGain

Our next goal is to evolve AgentGain from a hackathon prototype into a platform for trustworthy agent-to-agent commerce.

We plan to support autonomous purchasing agents, expand evaluation across different AI models and task types, strengthen OneIntent's duplicate-purchase detection, and introduce production-grade authentication and payment reliability.

Ultimately, we want AgentGain to become a trust layer for AI capability purchases — where agents can buy intelligently, sellers can prove their value, and buyers pay for verified results.