Chrome AI — Web Page Analyzer & Form Filler

Chrome AI — Web Page Analyzer & Form Filler Inspiration

The inspiration came from how much time we spend browsing and filling out repetitive online forms — from job applications to account sign-ups. I wanted a smart browser assistant that could understand webpages like a human, summarize their content, and automatically fill forms using contextual clues rather than hardcoded field names.

When I learned about Chrome’s new on-device AI capabilities (Summarizer, Translator, and future DOM analysis APIs), I wanted to explore how far we could go without relying on cloud AI services — keeping everything local, private, and fast.

What it does

This Chrome extension integrates with Chrome’s built-in AI features to perform several intelligent webpage tasks:

🧠 Summarizes the visible content of a webpage using the Chrome Summarizer API (if available).

🕵️ Analyzes DOM elements to identify and label forms intelligently — linking fields to their corresponding s.

✍️ Auto-fills forms using a local JSON data file that maps field labels (e.g., “Name”, “Email”) to user data.

🌍 Translates visible text using the built-in Translator API.

📊 User may ask anything in the Extension Chat box with build-in Chrome AI.

If Chrome’s native AI is unavailable, the extension gracefully falls back to lightweight JavaScript heuristics and message-passing logic to achieve similar behavior.

How we built it

Extension architecture

manifest.json defines permissions (activeTab, scripting, storage) and registers the popup UI and content scripts.

A popup UI (React + TailwindCSS or simple HTML/JS) provides user controls for analysis, summarization, and autofill.

A content script injects logic into web pages to traverse the DOM, collect text nodes, and locate input fields.

AI integration

Used Chrome’s experimental AI APIs (chrome.ai.summarizer, chrome.ai.translator) for local inference when supported.

Designed an abstraction layer:

const AI = { summarize: async (text) => { if (window.ai?.summarizer) return await ai.summarizer.summarize(text); return simpleTextSummary(text); // fallback }, };

Form analysis & filling

Parsed the DOM with querySelectorAll('input, textarea, select').

Challenges we ran into

🔒 Limited documentation on Chrome’s new AI APIs — they’re still experimental and not widely supported.

🧩 Context detection — determining which or nearby text corresponds to a specific form field required careful DOM traversal and heuristics.

🌐 Fallback handling — ensuring smooth operation when AI features weren’t available.

🪶 Balancing privacy and usability — keeping all data local without any cloud calls meant re-implementing logic that AI models usually handle easily.

Accomplishments that we're proud of

Built a fully functional Chrome extension that uses AI locally, without any external API keys.

Designed a hybrid approach that works across multiple Chrome versions (with or without AI APIs).

Implemented a context-aware form filler that genuinely “understands” nearby text rather than relying on field names.

Learned how to bridge UI, DOM scripting, and AI inference within the constraints of Chrome Extensions.

What we learned

Deep understanding of Chrome’s extension architecture, messaging system, and permissions model.

Hands-on experience using emerging Chrome AI APIs — Summarizer, Translator, and eventually Rewriter.

Improved knowledge of DOM analysis algorithms and how to rank relationships between nodes.

How to simulate user interaction safely through event dispatching and mutation observation.

What's next for Chrome AI — Web Page Analyzer & Form Filler

🚀 Add AI-based intent detection — e.g., auto-recognize “payment”, “signup”, or “survey” forms.

🤖 Integrate with Chrome’s on-device Rewriter and Embeddings APIs once public.

🧩 Expand translation and summarization to work in multiple languages simultaneously.

💾 Add per-site customization and form templates for recurring tasks.

🧠 Possibly train a lightweight local model to better predict field matches using vector embeddings.

The source code repo will be shared to public after 10/31

Built With

• ai
• autofill
• build-in
• chrome
• javascript