About the Project

What Inspired Me

For years, SEO has been a single-player game focused entirely on Google. But the sudden rise of generative AI search engines—like Perplexity, ChatGPT, and Claude—is fundamentally shifting how users discover content.

I realized there was a massive gap in the market: traditional SEO tools simply don’t measure how well content is optimized for Large Language Models (LLMs). I wanted to build a platform that treats AI search as a first-class citizen. My goal was to unify traditional SEO and Generative Engine Optimization (GEO) into a single, concrete platform that turns vague AI research into an actionable, quantitative score. That is how Clonvo was born.

What I Learned

Building a production-ready application taught me how to bridge the gap between abstract AI theory and deterministic software engineering:

  • Decoding GEO Principles: I learned how AI citation patterns drastically differ from traditional keyword rankings. I had to figure out how to programmatically measure things like citable answer block density, link-free capsules, and AI crawler accessibility (like llms.txt).
  • Mastering the Modern Stack: I deepened my expertise in Next.js 16 and React 19, specifically leveraging React Server Components and Server-Sent Events (SSE) to handle long-running, asynchronous tasks without blocking the UI.
  • Resilient Data Pipelines: I learned how to orchestrate multiple external APIs in parallel and handle partial failures gracefully so that a single timeout doesn't crash the entire user experience.
  • Database Architecture & Gating: I learned how to design a scalable database schema using Drizzle ORM to soft-gate premium features—storing complete analysis data on the backend, but safely stripping restricted fields at the API layer for free-tier users.

How I Built It

Clonvo is not a fragile prototype; it is built on a resilient, scalable Next.js architecture. The core pipeline follows a strict Crawl → Parse → Audit → Score → Recommend flow:

  1. The Crawler Engine: An SSRF-protected fetcher that validates URLs, blocks private IP ranges, enforces a 5MB payload limit, and securely handles redirects.
  2. Data Extraction: Using Cheerio, the system parses the raw HTML to extract over 100 data points (meta tags, heading hierarchies, schema.org JSON-LD, etc.).
  3. Parallel API Execution: To keep the tool lightning-fast, I use Promise.allSettled to simultaneously hit Google PageSpeed, Open PageRank, the Wayback Machine CDX, and DataForSEO.
  4. The Five Audits: The extracted data is run through 95 distinct checks categorized into five pillars: On-Page, Technical, Content, GEO, and Authority.
  5. Weighted Scoring Algorithm: I engineered a composite scoring system to give users an at-a-glance health metric. The math behind the overall score is calculated as:

$$Score_{overall} = (0.25 \times OnPage) + (0.25 \times Technical) + (0.20 \times Content) + (0.15 \times GEO) + (0.15 \times Authority)$$

  1. Actionable Recommendations: Every failed check dynamically generates a prioritized recommendation card, sorted by a custom severity and impact matrix.
  2. Real-Time UI: Instead of making users wait at a loading screen, I implemented Server-Sent Events (SSE) to stream the analysis progress live to the client dashboard.

Challenges I Faced

Building a tool that crawls arbitrary, user-submitted URLs introduces massive security and stability risks. Here is how I solved the core engineering hurdles:

The Challenge My Approach
SSRF Vulnerabilities I built a custom validation layer that strictly blocks localhost/private IPs and aggressively re-checks DNS resolution on every single redirect hop.
Third-Party API Instability Since Clonvo relies on multiple external APIs, a timeout could ruin an audit. I wrapped all external calls in a Promise.allSettled block, allowing the core Cheerio parser to finish the audit even if an external API degraded.
Quantifying "GEO" Turning qualitative AI research into code was tough. I had to build custom parsers to count specific HTML patterns (like 20–35 word "capsules" and structured data arrays) to map abstract AI concepts into a strict 0–100 score.
Plan Gating Logic To allow users to upgrade and immediately see historical premium data, I designed the backend to always run a full 95-point scan. A custom stripRestrictedData() utility ensures Free users only receive partial JSON payloads until they upgrade via Polar.
Abuse & Rate Limiting To protect my database and API quotas, I implemented strict user-based rate limiting (5 scans/minute) alongside a 30-day rolling scan quota tied to their subscription tier.

Built With

  • anthropic
  • app-router
  • bun
  • cheerio
  • clerk
  • css
  • dataforseo
  • drizzle
  • google
  • icons
  • insights
  • machine
  • migrations
  • neon
  • next-themes
  • next.js
  • openai
  • openpagerank
  • orm
  • pagespeed
  • phosphor
  • polar
  • postcss
  • postgresql
  • radix
  • react
  • recharts
  • shadcn/ui
  • svix
  • tailwind
  • technology
  • typescript
  • ui
  • wayback
  • zod
Share this project:

Updates

Submission history