Inspiration
What it doesInspiration
As a manual QA engineer, I spend my days hunting accessibility and UI bugs by hand. I wanted an agent that could do the first pass instantly — on any webpage — without sending data to a server or paying per scan. Firebase AI Logic's hybrid inference was the perfect fit: run the audit on-device with Gemini Nano when available, and fall back to the cloud only when the device can't do it.
What it does GhostQA Local-First Sentinel is a web app (and Chrome extension) that scans any webpage's DOM for accessibility and UI issues — missing alt text, unlabeled form inputs, contrast problems, and more. It audits each element with AI, shows ON-DEVICE or CLOUD badges with latency, tracks money saved by running locally, and generates a unified-diff auto-fix patch for confirmed issues via cloud AI. A built-in "Test AI Paths" diagnostic pings both inference paths so you always know which one is live.
How I built it Frontend: React + Vite, deployed on Railway (free tier). AI: Firebase AI Logic SDK (v12.5.0+) with InferenceMode.PREFER_ON_DEVICE for triage (Gemini Nano via Chrome Prompt API) and ONLY_IN_CLOUD for deep fixes (Gemini models via the Gemini Developer API). The app warms up Nano with a one-word prompt, runs concurrent audits with per-call timeouts, and automatically rescues to the cloud (trying several Gemini models in sequence) when on-device is slow or unavailable. Firebase App Check protects the AI Logic API; the debug token is localhost-only so the public deployment stays clean. Challenges I ran into The on-device model reports "ready" but the first inference can take 15–35s to load weights into memory — and Nano processes prompts strictly sequentially, so parallel calls queue up and time out. I fixed it with a warm-up call, a fast 8s on-device timeout, and automatic cloud rescue. A model name suggested by an API error message turned out to be wrong for the Firebase AI Logic backend. I verified the real model list against the official docs and built a multi-model fallback chain instead of trusting one name. Railway removed the "Deploy latest commit" command mid-project, so deployments are done by pinning the service to a verified commit. Accomplishments that I'm proud of End-to-end runtime proven on a real browser: a cloud AI audit returns a real accessibility finding in ~4s, and the deep-fix generator produces a correct unified diff that adds the missing aria-label. Honest status reporting: the UI shows exactly which inference path answered (ON-DEVICE / CLOUD / AI ERROR) and how long it took — no fake badges. What I learned Hybrid inference is not "set and forget": on-device availability has states (readily / downloadable / unavailable) that must be handled explicitly, and the first inference is a cold start. Always verify an AI assistant's API claims against the official docs — even model names. A diagnostic button that pings each path separately is worth its weight in gold when debugging. What's next Ship the Chrome extension (MV3) so the Sentinel can audit any open tab in one click. Add more checks (SEO meta, heavy assets, layout overflow on mobile). Let users download fixes as a .kiro/specs/ghost-fixes.md file for their IDE agent.
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for GhostQA Local-First Sentinel
Built With
- chrome
- firebase-ai-logic
- firebase-app-check
- gemini
- gemini-nano
- github
- javascript
- railway
- react
- vite