AgenticBridge

In a future with AI agents, most websites aren’t ready. AgenticBridge helps make websites more AI-friendly by scoring and translating pages.

Comment

AgenticBridge: Making Websites Friendly for AI Agents

Inspiration

During spring break, we went to Los Angeles and used an AI assistant to help plan our trip. We asked it to schedule a visit to the Getty Center, but it gave the wrong opening hours. The website actually shows the correct hours, but the AI couldn’t read them because the information was loaded through JavaScript.

That made us realize that many websites aren’t designed for AI agents to understand. We were also inspired by tools like Semrush, which help websites improve their visibility and structure. This made us wonder: what if there were a tool that helps websites become easier for AI agents to understand?

What It Does

  • Crawls multiple pages of a website and scores how AI-friendly the content is
  • Points out specific areas that make the site difficult for AI agents to read
  • Translates webpages into a cleaner, AI-friendly version by removing layout clutter and keeping key information

Technical Implementation

Frontend

  • Built with React, TypeScript, and Tailwind CSS
  • Handles user input, analysis flow, and results display

Backend

  • Uses Supabase Edge Functions for processing and data storage
  • Includes caching, user accounts, and search history

Crawling & Analysis

  • Firecrawl used to discover and scrape webpages
  • Gemini evaluates pages using a structured scoring rubric
  • Results aggregated into a site-wide score

AI Analysis (Scoring System)

  • Gemini evaluates pages using a 5-category rubric
  • Categories: Access Barriers, Content Accessibility, Structure Clarity, Navigation Clarity, Machine Readability
  • Returns structured scores and combines them into a final site score

Translation Feature

  • Claude converts webpages into clean, AI-friendly Markdown
  • Removes layout noise and keeps core content

Scoring System

5 categories: Access Barriers, Content Accessibility, Structure Clarity, Navigation Clarity, Machine Readability

What Each Category Looks At

  • Access Barriers: Checks for things like logins, paywalls, or CAPTCHAs that block access.
  • Content Accessibility: Checks if the main content is easy to read and not hidden behind scripts or missing text.
  • Structure Clarity: Looks at headings, sections, and whether the page is organized in a clear way.
  • Navigation Clarity: Sees if links, menus, and page flow are easy for AI to follow.
  • Machine Readability: Looks at technical details like metadata and structured data for AI processing.

Why Each Category Has Its Weight

The scoring is split into 90% vs 10%. The first four categories (90%) decide whether AI can actually access and understand the content, while the last 10% focuses on how well the quality of the content is. Accessing the content (90%) is way more important that the quality of the content (10%).

  • Access Barriers (30 pts, most important): If AI cannot access the page, everything else fails.
  • Content Accessibility (25 pts, second): After access, the content must be readable and visible.
  • Structure Clarity (20 pts, third): Clear structure helps AI understand meaning and context.
  • Navigation Clarity (15 pts, fourth): Helps AI move across pages and find related content.
  • Machine Readability (10 pts, least important): Improves quality, but only matters after content is accessible.

Challenges

Incomplete Crawling (Subdomains): At first, our crawler did not properly include subdomains, which caused important pages to be missed. This led to incomplete and less accurate analysis results.

Finalizing the Scoring Rubric: We went through 9 versions of our rubric because we used ideas from multiple AI models to make it unbiased. It was challenging to combine different approaches into one fair and consistent system.

What We've Built

  • Built a working scoring system that evaluates AI-friendliness across multiple categories
  • Built a webpage translation feature that converts pages into AI-friendly Markdown
  • Tested the translated pages against our rubric and saw scores improve by 20–30 points, validating both the scoring system and the translation approach

Impact

  • Empty rooms, real losses — a hotel with 10 unsold rooms every night isn't just inconvenient, it's money gone; if an AI agent could actually read the site, it could match those rooms with last-minute travelers looking for a deal; the hotel fills the room, the traveler saves money, it is a win-win situation
  • Better reading = better recommendations — the more accurately AI agents understand what's on a website, the better they get at connecting people to what they actually need, whether that's a hotel room, a open restaurant table, or an item back in stock
  • A new way for businesses to get discovered — as more people use AI assistants to make decisions, being AI-readable is becoming just as important as being Google-searchable

What We Learned

  • No single AI model is good at everything; different models have different strengths
  • Used Gemini for scoring and Claude for translation, making sure they worked well together as a system
  • Got better at choosing the right tools and combining them into one cohesive pipeline

Future Work

  • Just as websites have "www." for desktop and "m." for mobile, we believe "a." versions optimized for AI agents could become a standard
  • Plan to build a feature that automatically generates AI-friendly versions of entire websites
  • Already ahead of the concept stage; the webpage translation feature is live and working on our project link today

Built With

Share this project:

Updates