Perry

Inspiration

When we saw Meta release Muse and the growing capabilities of AI agents to complete everyday tasks on our behalf, we started thinking about a bigger question: What happens when the internet's primary users are no longer humans, but AI agents?

Imagine a world where you don't spend 30 minutes browsing stores, comparing prices, filtering products, and navigating checkout pages. Instead, you tell your AI assistant, "Find me a pair of black running shoes in size 8 for under $120," and it handles everything.

We're getting closer to that reality. But there's one fundamental problem: the internet was designed for humans, not agents.

Today's e-commerce websites are built around visual layouts, navigation menus, product cards, and endless pages. These interfaces make sense to us, but AI agents have to repeatedly interpret webpages, navigate between products, and extract information that's often already available in a store's underlying database.

We realized that as AI agents become more intelligent, the bottleneck won't necessarily be the agents themselves. It'll be the infrastructure they're forced to interact with.

Instead of building another AI shopping assistant, we wanted to build something for the assistants themselves.

That's how Perry was born.

What It Does

Perry transforms traditional e-commerce websites into shopping experiences optimized for AI agents.

We built two complementary approaches that address both the current internet and the agent-first internet we envision.

1. Agent-Optimized Websites

Not every AI agent can interact directly with a store's backend. Many still rely on browsing websites, just like humans.

Perry observes an AI agent completing a real shopping task, analyzes its behavior, and identifies unnecessary steps. Using the Factory Droid SDK, Perry then autonomously modifies the storefront's source code to make information more accessible to agents.

For example, our shopping agent was tasked with finding black running shoes in size 8, in stock, under $120. On the original website, it had to navigate through multiple product pages to determine availability. Perry identified this inefficiency and used Droid to modify the search interface so agents could access critical product information directly.

Crucially, Droid isn't just helping us write code during development. It's an autonomous component of Perry itself, generating actual source-code changes in response to observed agent behavior.

2. Agent-Native Commerce

While developing our first approach, we started questioning something more fundamental: Why are we optimizing webpages for AI agents when agents might not need webpages at all?

Every e-commerce store already has a structured product catalog, inventory information, and shopping cart functionality. A website is essentially a visual representation of that underlying data.

So we built a second approach that skips the visual interface entirely.

Perry uses Factory Droid to generate an agent-native shopping interface from the store's existing catalog and cart API. Through Model Context Protocol (MCP), AI agents can directly search products, check availability, and add items to real shopping carts using structured commands.

Instead of reading thousands of tokens of webpage content, an agent receives only the relevant product information it needs.

We demonstrated this integration with Claude Desktop, allowing an external AI agent to interact with Perry's generated shopping tools and add the correct product to a real store cart.

How We Built It

We built Perry using TypeScript, Next.js, React, Playwright, Anthropic, Factory Droid, and MCP, combining autonomous browser agents, code generation, benchmarking, and real shopping cart verification.

Our system follows a continuous optimization workflow:

1. Observe: A Playwright-powered AI agent completes a shopping task while Perry records its actions, token usage, execution time, and cost
2. Diagnose: Perry analyzes the execution trace to identify inefficient navigation, redundant actions, and missing product information
3. Optimize: Factory Droid generates targeted source-code improvements in an isolated environment, with restrictions on which files it can modify
4. Generate: Droid can also create structured shopping commands from the underlying catalog and cart API
5. Verify: Perry checks the actual shopping cart to confirm that the agent selected the correct product
6. Compare: Our dashboard benchmarks all three approaches using equivalent shopping tasks

One of our most important design decisions was making cost per verified successful task our primary metric. Reducing tokens or clicks means nothing if an agent ultimately purchases the wrong item. We wanted to optimize efficiency without sacrificing accuracy.

Results

We ran 15 live tests, with five runs for each approach. The agent successfully added the correct product to the cart in every run.

Metric	Original website	Optimized website	Agent interface
Verified completion	5/5	5/5	5/5
Average tool calls	6.6	4.0	2.0
Average tokens	32,594	22,469	3,732
Cost per successful task	$0.00358	$0.00246	$0.00047

Compared to the original website, Perry's agent-native interface achieved:

• 88.6% fewer tokens
• 86.9% lower cost per successful task
• 69.7% fewer tool calls
• 100% verified task completion across all 15 tests

Even our website optimization approach reduced token usage by approximately 31%, demonstrating that both approaches meaningfully improve agent efficiency.

Changing how agents access information had a far greater impact than simply improving how they navigate it.

Challenges and What We Learned

Our biggest challenge was figuring out what actually makes agent-website interactions more efficient.

Initially, our entire approach centered on restructuring existing websites. We assumed that if we could reorganize product information, simplify navigation, and expose important details earlier, agents would shop more efficiently.

And that worked. Our initial optimizations reduced unnecessary browsing steps and execution costs.

But after deeper research into how autonomous agents interact with websites, emerging agent protocols, and where their computational costs actually come from, we realized we were solving only part of the problem.

Even on an optimized website, an agent is still interpreting HTML, navigating pages, and reconstructing information that already exists in structured form.

That led to our biggest realization of the hackathon:

Why teach AI agents to navigate human interfaces more efficiently when we can give them an interface designed specifically for machines?

We pivoted from focusing exclusively on website optimization to building an agent-native commerce layer as well.

This introduced an entirely new set of technical challenges. We had to figure out how to expose real product data through reliable commands, support external agents through MCP, safely generate those commands using Factory Droid, and verify that an agent's actions actually modified the store's cart correctly.

Ultimately, our biggest lesson was that the most effective way to improve AI agent performance isn't always to build a smarter agent. Sometimes, it's to redesign the environment the agent operates in.

What's Next

Perry started as a tool to optimize shopping websites. We now see it as the beginning of something much bigger: an infrastructure layer for an agent-first internet.

We want to expand Perry beyond our demonstration storefront to support real e-commerce platforms, automatically generate agent-native interfaces for different stores, and develop standardized ways for agents to discover and interact with businesses.

Longer term, we envision a world where online businesses offer two interfaces: a visual experience for humans and a structured, action-oriented interface for AI agents.

As agents take on more of our everyday tasks, websites shouldn't have to remain the middleman between an agent and the information it needs.

The next generation of the internet won't just be browsed by agents. It'll be built for them.

Built With

• anthropic
• apify
• droid
• factory
• mcp
• next.js
• node.js
• playwright
• react
• typescript