AgentReady — AI Commerce Infrastructure
Inspiration
E-commerce is evolving. Traditionally, customers visit online stores, browse product pages and manually compare products. With the emergence of AI shopping assistants and autonomous agents, customers are increasingly able to describe what they need and let AI systems help discover and compare suitable products.
However, there is an important infrastructure problem: most existing businesses are not ready for AI-powered commerce.
Many merchants already have websites containing hundreds or thousands of products, but their product information is scattered across HTML pages, descriptions, tables, metadata and inconsistent formats. Product names, prices, brands, specifications and availability are represented differently across websites.
An AI agent may be capable of understanding a product description, but reliably discovering, comparing and accessing information across many inconsistent merchant websites is a much harder problem.
We wanted to solve this without asking businesses to rebuild their existing websites or migrate to an entirely new commerce platform.
That idea led to AgentReady.
What is AgentReady?
AgentReady is an AI-powered commerce infrastructure platform that transforms existing merchant websites into structured, validated, manageable and AI-accessible product catalogs.
It acts as a bridge between traditional e-commerce websites and the emerging ecosystem of AI shopping agents.
A merchant connects an existing website. AgentReady crawls its relevant pages, extracts product information, converts it into a standardized product structure, validates the extracted data and provides tools for managing and distributing the catalog.
Approved product information can then be accessed through REST APIs, machine-readable product feeds and Model Context Protocol (MCP) tools.
For example, imagine a customer asking an AI shopping agent:
"Find me a laptop with 16 GB RAM and at least 512 GB SSD under ₹60,000."
Instead of requiring every merchant to build a separate AI integration, AgentReady provides a structured catalog and discovery interface that an agent can query.
Our goal is to make existing businesses discoverable and understandable to AI-powered commerce systems.
How We Built It
We developed AgentReady across five major phases.
Phase 1 — AI Catalog Generator
We built a website crawling and product extraction engine.
Merchants can connect their existing websites, and AgentReady discovers relevant product pages while respecting crawling rules and applying security restrictions.
The extraction engine uses a hybrid approach:
- Deterministic extraction using JSON-LD, Microdata, metadata, HTML heuristics, tables and product cards.
- Groq-powered LLM extraction when deterministic methods cannot reliably extract the required information.
- Structured product normalization and validation.
- Field-level source evidence and provenance.
- Duplicate detection and merchant review.
- Product approval and structured catalog exports.
A key design decision was to use deterministic extraction first and AI as a fallback. This reduces unnecessary LLM calls and avoids depending entirely on generated output.
We also implemented protections against Server-Side Request Forgery (SSRF), unsafe redirects, excessive crawling and resource abuse.
Phase 1.5 — AI Extraction Reliability
We strengthened the Groq integration using direct HTTP requests through HTTPX.
We implemented structured output handling, extraction validation, token and cost logging, and graceful fallback when AI extraction fails.
We also tested behavior with invalid API credentials to ensure the application could fall back to deterministic extraction instead of failing completely.
Phase 2 — Commerce Management
Extracting products is only the beginning. Merchants also need to manage their catalogs.
We developed a commerce management layer with:
- Product creation, editing, approval and lifecycle management.
- Pricing and price history.
- Inventory tracking and stock transactions.
- Product variants.
- Categories and custom attributes.
- Bulk operations.
- Scheduled rescanning.
- Website synchronization and conflict resolution.
- Catalog API keys and access controls.
- Audit logs and merchant dashboards.
An important principle throughout this phase was preserving merchant control. Website rescans should not silently overwrite merchant-managed information.
We also implemented Shopify and WooCommerce integration foundations, with live store connections requiring merchant credentials.
Phase 3 — AI Commerce Distribution
This phase transformed AgentReady from a catalog management application into an AI commerce infrastructure platform.
We developed a standardized public product representation and agent discovery APIs.
Key capabilities include:
- Product search and filtering.
- Product and variant discovery.
- Pricing and availability retrieval.
- Product comparison and recommendations.
- Merchant discovery.
- Cart and checkout handoff foundations.
- Versioned product feeds.
- JSON, JSON Lines, CSV, XML and Schema.org JSON-LD formats.
- MCP server and agent tools.
- Webhooks and distribution analytics.
- Public distribution controls and revocable feed access.
We also implemented a discovery capability document at /.well-known/agentready.json.
The platform exposes only approved public product information. Internal notes, cost prices, private inventory quantities and source evidence are excluded from public serialization.
External platform onboarding and live integrations remain subject to their respective credentials, permissions and approval processes.
Phase 4 — Commerce Intelligence
We then added an intelligence layer to help merchants understand and improve their catalogs.
This includes:
- Catalog quality scoring.
- Product-level quality reports.
- Missing attribute and information detection.
- Product discovery insights.
- Zero-result search analysis.
- Evidence-based recommendations.
- AI-assisted product description and attribute improvements.
- Merchant-approved AI changes.
- Pricing and inventory intelligence.
- Freshness and anomaly detection.
- PDF, CSV and JSON reports.
- Notifications and audit history.
The AI assistant uses catalog information and supporting evidence rather than inventing business metrics or unsupported product specifications.
Phase 5 — Final Production Readiness
The final phase focused on making the platform deployable and dependable.
We reviewed application security, database integrity, background jobs, API reliability, frontend behavior, performance, deployment configuration, observability, documentation and regression testing.
The objective was to make the complete product workflow operate reliably rather than simply adding more features.
Technical Architecture
AgentReady uses a modular full-stack architecture:
Frontend-->
- React
- TypeScript
- Tailwind CSS
Backend-->
- Python
- FastAPI
- SQLAlchemy
- Alembic
Database and background processing-->
- PostgreSQL
- Redis
- Celery
- Celery Beat
AI and extraction-->
- Groq API
openai/gpt-oss-120b- HTTPX
- BeautifulSoup
- Structured extraction and validation
Commerce interfaces-->
- REST APIs
- MCP
- JSON and JSON Lines
- CSV and XML
- Schema.org JSON-LD
- Webhooks
Infrastructure-->
- Docker
- Docker Compose
- Nginx
- Render deployment
The architecture separates product discovery and data preparation from agent-facing product access. Crawling and extraction happen through background processing, while merchants and authorized agents interact with the prepared catalog through APIs and interfaces.
Challenges We Faced
Building AgentReady involved several engineering challenges.
- Extracting products from inconsistent websites
Different websites use different HTML structures, field names and product representations. A single extraction strategy was not sufficient.
We addressed this using multiple deterministic extraction strategies, followed by Groq-based extraction when necessary.
- Preventing AI hallucinations
LLMs can generate plausible-looking product information that is not present on the original webpage.
We implemented field-level validation against source content, rejected unsupported prices and other claims, and retained evidence for extracted values.
- Building a secure crawler
A merchant-supplied URL cannot be trusted automatically. The crawler could potentially be abused to access internal network resources.
We implemented URL validation, SSRF protections, redirect validation, crawling limits and resource restrictions.
- Preserving merchant control
An automated rescan should not overwrite information that a merchant has manually corrected.
We introduced merchant-managed data, source provenance, conflict handling and explicit approval workflows.
- Designing safe agent-facing distribution
Not every field stored in a merchant catalog should be exposed to external agents.
We designed a separate public product representation with explicit controls over published products and private information.
- Building a complete asynchronous system
Website crawling, feed generation and intelligence calculations can take time. Running everything directly inside normal HTTP requests would create reliability and responsiveness problems.
We used Redis and Celery for background processing and job coordination.
- Building intelligence without fabricated metrics
Catalog quality and discovery insights must reflect actual catalog data and recorded agent interactions. We avoided presenting invented sales, revenue or conversion figures as real business intelligence.
What We Learned
Building AgentReady taught us that creating an AI-powered product is not just about connecting an LLM to an application.
We learned how to design complete data pipelines, build secure crawlers, work with asynchronous processing, create structured extraction systems, validate AI-generated information, manage relational data, expose versioned APIs and implement controlled agent access.
One of our biggest takeaways was that AI reliability depends heavily on the quality of the data and infrastructure surrounding the model.
We also learned to treat security, provenance, merchant approval and data privacy as core product requirements rather than features to add later.
What Makes AgentReady Different?
AgentReady is designed as an end-to-end commerce enablement layer rather than just a product scraper or a product feed generator.
It brings together:
- Website crawling and catalog generation.
- Product normalization and validation.
- Merchant-side catalog and inventory management.
- Controlled AI commerce distribution.
- REST APIs, feeds and MCP access.
- Catalog intelligence and AI-assisted optimization.
Instead of requiring businesses to rebuild their existing stores for AI commerce, AgentReady helps prepare the product information they already have.
Our vision is simple: Make every existing merchant catalog ready for the next generation of AI-powered commerce.
Built With
- api
- artificial
- beautiful-soup
- celery
- css
- docker
- e-commerce
- fastapi
- groq
- intelligence
- language
- large
- machine-learning
- mcp
- models
- nginx
- postgresql
- python
- react
- redis
- render
- rest
- tailwind
- typescript
- webscraping