AgentProof

Making online stores ready for AI shoppers.

Inspiration

The next customer your online store loses might never visit your website.

For decades, e-commerce has been built around humans. Merchants spend enormous effort making sure their websites work across different devices and that customers can shop without friction.

But a new kind of customer is emerging: AI agents that shop on our behalf.

And these agents don't necessarily see websites the way we do.

One agent might navigate through accessibility labels. Another might interpret screenshots. The same storefront can behave completely differently depending on who's interacting with it.

A website can pass every traditional test and still fail an AI customer.

An agent might struggle to find a product or fail to complete a purchase. To the merchant, everything looks normal. Their website works, yet they have no way of knowing whether AI agents can reliably shop there.

We realized that as commerce becomes increasingly agent-driven, businesses will need an entirely new way to test their storefronts.

That led us to a simple question:

If we test websites for every browser and device, why aren't we testing them for AI agents?

That's why we built AgentProof.

What it does

AgentProof is a testing and repair platform that helps merchants identify and fix problems preventing AI agents from shopping correctly.

A merchant enters their store URL and defines a shopping mission, such as finding a medium black hoodie under $60 and adding it to the cart.

AgentProof sends ABGS (our four specialized agents) in to complete the mission:

Alpha — Semantic: Navigates using accessibility information.
Beta — Vision: Interacts through screenshots and screen coordinates.
Gamma — Commerce: Tests product catalogs and cart APIs directly.
Sigma — Hybrid: Combines visual and semantic navigation.

Each agent interacts with the store differently, revealing problems the others might miss. Our independent verification engine checks the actual cart against the customer's requirements. Merchants can inspect agent activity and failures through a live dashboard.

AgentProof then goes a step further. By connecting to the merchant's GitHub repository, it can identify the relevant code, generate a targeted fix, and rerun the mission to verify the repair. Developers receive a reviewable patch or draft pull request.

How we built it

We built AgentProof with Next.js, React, TypeScript, and Fastify, using SQLite for persistence and Playwright for browser automation.

Our semantic, vision, and hybrid agents use LLMs through an architecture supporting Anthropic and xAI. Gamma uses deterministic TypeScript and HTTP requests to test commerce APIs.

We also built Northwind Threads, a synthetic storefront with five deliberately planted bugs to test our system.

Finally, we integrated GitHub with an AI-powered repair pipeline that uses failure evidence to generate code patches, validate them, and retest the storefront.

Challenges we ran into

One of our biggest challenges came from our original approach. We initially built a single AI shopping agent to test storefronts, but quickly ran into a problem: when a purchase failed, we couldn't tell whether the agent had made a mistake or the website was at fault.

That realization led us to redesign AgentProof around four specialized agents, each testing a different way of interacting with a store. By comparing their results, we could isolate failures much more effectively. If a visual agent succeeded where a semantic agent failed, for example, we had evidence that the issue might lie in the website's accessibility information rather than the shopping task itself.

Accomplishments that we're proud of

Our biggest accomplishment was finding a way to make Alpha, our lightweight semantic agent, work across a much wider range of websites. By reducing our reliance on expensive vision-based agents, this approach has the potential to cut AI shopping agent costs by up to 25×, making agentic commerce significantly more affordable at scale.

What we learned

AI shopping reliability isn't just an AI problem. It's also a website compatibility problem.

A website can work perfectly for humans while communicating incorrect information to an AI agent. Different agents can encounter entirely different failures on the same storefront.

We also learned the importance of independent verification. An agent claiming success means little unless its actions actually satisfy the customer's request.

Combining AI-driven exploration with deterministic testing allowed us to move beyond identifying problems toward validating real solutions.

What's next for AgentProof

We want to bring AgentProof to real e-commerce platforms and expand testing to more complex shopping experiences.

Our next major goal is continuous agent compatibility testing. Imagine a merchant updating their storefront and automatically testing whether AI shoppers can still complete purchases before deploying the changes.

Eventually, we envision agent compatibility becoming a standard part of e-commerce development, just like browser compatibility testing.

As AI agents become customers, merchants need to know whether their stores are ready. AgentProof helps them find out.