NovaGuard

A multi-agent pipeline using Nova Act, Nova 2 Lite, and Nova 2 Sonic to automatically crawl websites, detect WCAG 2.2 violations, generate code fixes, and apply them — with human-in-the-loop approval.

Comment

Inspiration

Web accessibility isn't optional — it's a legal requirement and a moral imperative. Yet 96% of the top million websites fail basic WCAG compliance checks. Manual audits are slow, expensive, and inconsistent — a single audit can take weeks and cost thousands of dollars. Meanwhile, millions of users with disabilities face barriers every day.

I asked: what if AI agents could audit, fix, and verify accessibility issues autonomously — in minutes instead of months?

What it does

NovaGuard is an AI-powered accessibility compliance agent that runs a fully automated 6-stage pipeline:

  1. Crawl — Nova Act navigates the target site, captures screenshots, extracts the full DOM, and tests keyboard navigation
  2. Analyze — Nova 2 Lite performs multimodal analysis on screenshots + DOM to identify WCAG 2.2 Level AA violations
  3. Fix — Nova 2 Lite generates minimal, targeted HTML/CSS patches for each violation
  4. Approve — Human-in-the-loop review with side-by-side before/after diffs
  5. Apply — Nova Act applies approved patches directly in the browser
  6. Verify — Nova Act re-checks fixed elements and captures proof screenshots

Users can also interact with Nova 2 Sonic via voice — asking questions about findings, requesting explanations of WCAG criteria, or triggering fix actions conversationally.

Everything streams in real time to a live dashboard with a visual pipeline stepper, findings cards with severity badges, a code diff viewer, and before/after screenshot comparison.

How I built it

Backend — Python with FastAPI and WebSockets. The 6-node pipeline is orchestrated using the Strands Agents SDK GraphBuilder. Each node is a focused agent:

  • Nova Act powers the Crawl, Apply, and Verify nodes — running real Chromium browser automation with targeted act() calls for page navigation, keyboard testing, element inspection, and fix application
  • Nova 2 Lite (amazon.nova-lite-v2:0) powers the Analyze and Fix nodes — ingesting base64-encoded screenshots alongside raw DOM text to detect violations and generate code patches
  • Nova 2 Sonic provides the voice interface via Strands BidiAgent for real-time conversational interaction

Run state is managed in-memory with asyncio Queues streaming events over WebSocket connections. Screenshots and DOM snapshots are stored per-run on disk.

Frontend — React 19 + TypeScript + Vite + Tailwind CSS + Framer Motion. A single-page dashboard with:

  • Project management with multi-project support
  • Real-time event timeline fed by WebSocket
  • Color-coded pipeline stepper showing which agent is active
  • Findings panel with severity badges and WCAG criteria references
  • Side-by-side diff viewer for reviewing proposed fixes
  • Before/after screenshot comparison for verification
  • Voice panel with Nova Sonic integration

Challenges I ran into

Browser automation reliability — Nova Act interacts with real web pages, which are inherently unpredictable. Instead of one monolithic act() prompt, we designed 4 focused, deterministic calls per crawl (page load, keyboard navigation, interactive elements, form inspection), which dramatically improved consistency.

Multimodal prompt engineering — Getting Nova 2 Lite to output well-structured JSON findings from screenshots + DOM required extensive iteration. The model needed careful prompting to distinguish between visual issues (contrast, spacing) and structural issues (missing labels, heading hierarchy) and map each to the correct WCAG criterion.

Voice command boundaries — Nova Sonic initially auto-triggered fix and approval actions when users just asked conversational questions about findings. We solved this with strict command parsing rules in the system prompt — explicitly separating "explaining a finding" from "approving a fix."

Real-time state consistency — Streaming events from an async multi-agent pipeline through WebSockets while keeping the React UI consistent across project switches, run cancellations, and reconnections required careful state management and cleanup logic.

Accomplishments that I'm proud of

  • End-to-end autonomous pipeline — From URL input to verified fix with proof screenshots, the entire flow runs with a single click and one human approval step
  • Real multimodal analysis — Nova 2 Lite analyzes actual screenshots alongside DOM structure, catching visual issues that text-only tools miss entirely
  • Human-in-the-loop done right — The approval gate shows exactly what will change with code diffs and before/after previews, building trust without sacrificing automation
  • Voice-powered accessibility — Using Nova Sonic to explain accessibility findings conversationally makes the tool itself more accessible
  • Production-quality UI — A polished, real-time dashboard that looks and feels like a professional product, not a hackathon demo

What I learned

  • Multi-agent architectures shine when each agent has a narrow, well-defined role — trying to make one agent do everything leads to unreliable results. Six focused agents outperform one generalist
  • Multimodal AI catches what text-only analysis misses — color contrast issues, visual hierarchy problems, and layout-based accessibility barriers are only detectable when the model can actually see the page
  • Human-in-the-loop builds trust — fully autonomous fixing sounds impressive, but users need to see and approve changes before they go live. The approval gate is a feature, not a limitation
  • Event-driven architecture scales naturally — WebSocket streaming with typed events made it straightforward to add new pipeline stages and UI features without refactoring

What's next for NovaGuard

  • Multi-page deep crawling — Automatically follow links and audit entire sites, not just single pages
  • CI/CD integration — Run NovaGuard as a GitHub Action that blocks deploys with accessibility regressions
  • Fix learning — Build a knowledge base of successful fixes so the system improves over time and handles edge cases better
  • WCAG 2.2 Level AAA support — Expand beyond Level AA to cover the full specification
  • Accessibility report generation — Export PDF compliance reports with evidence screenshots for legal and audit teams
  • Browser extension — One-click auditing directly from the browser toolbar while browsing any site

Built With

Share this project:

Updates