FixTrace - Multi-agent frontend debugging platform

FixTrace is a live multimodal copilot that turns your UI screenshots, performance traces, and code into prioritized issues and ready‑to‑apply refactor patches through a real‑time Gemini voice session.

Comment

Inspiration

As a frontend developer, I was exhausted by the endless "screenshot gymnastics" and "trace archaeology" in my daily workflow. I'd take 5+ screenshots of broken layouts across breakpoints, record laggy animations, export massive Lighthouse traces, then paste everything into Slack or chat with vague explanations like "can you check this contrast?" or "why's my ngIf killing LCP?".

Gemini Live API's real-time voice + multimodal vision clicked perfectly: What if I could upload a screenshot/trace, hit record, and literally talk through fixes with an AI that sees my actual UI and code? FixTrace is that pair-programming partner I always wanted.

What it does

FixTrace is a live multimodal debugging copilot for frontend devs:

  1. Upload anything: UI screenshots, screen recordings, Lighthouse JSON, DevTools traces, code files, or GitHub repos
  2. Get instant analysis: Structured issues ranked by severity (accessibility scores, Core Web Vitals diagnosis, layout/contrast problems)
  3. Review code patches: Monaco Editor shows AI-generated unified diffs you can accept/edit/reject
  4. Talk it through: Real-time voice chat with a 3D avatar that lip-syncs responses via Gemini Live API
  5. Debate solutions: "Should I use signals or RxJS here?" "Why ngIf over OnPush?" "React approach vs Angular?"

Instead of static reports, you get a conversation about your actual code and UI problems.

How we built it

Frontend (my comfort zone)

  • Angular 21 standalone components + signals
  • Tailwind CSS v4 + DaisyUI for pixel-perfect styling
  • Three.js 3D avatar with real-time lip-sync from TTS audio
  • Monaco Editor (VS Code core) with diff viewer for patch review
  • Drag-and-drop file upload, GitHub repo cloning, issue cards

Backend (AI helped me survive)

  • Node.js 20 + Express 5 on Cloud Run
  • TypeScript services for UI analysis, perf analysis, code analysis, Imagen enhancement
  • WebSocket proxy for Gemini Live 2.5 Flash Native Audio (sub-second bidirectional streaming)
  • Google GenAI SDK (@google/genai v1.41+) with Vertex AI + API key dual auth

AI Pipeline

Screenshot/Trace/Code → Gemini 2.5 Flash → Structured issues + patches
"Make this sexier" → Imagen 4 → Design mockup + implementation
Talk to agent → Gemini Live API → 3D avatar lip-syncs response

Cloud Architecture

  • Cloud Run (backend container)
  • Cloud Storage (fixtrace-uploads-* buckets)
  • Firebase Hosting (Angular SPA)
  • Cloud Build + Terraform CI/CD

Challenges we ran into

  1. Backend as frontend dev: Express routing, WebSocket state, GCS signed URLs, error handling – all new territory. AI scaffolded ~70%, but debugging 500s was brutal.

  2. Live audio streaming: WebSocket chunk drops, latency spikes, microphone/backpressure coordination. Proxying Gemini Live through Express fixed most issues.

  3. 3D avatar lip-sync: Real-time PCM audio → phoneme detection → Three.js morph targets. Audio analysis was fiddly math.

  4. Multimodal context: Combining screenshot + trace + code into coherent analysis required careful prompt engineering and response parsing.

  5. Large file handling: 50MB Lighthouse traces needed chunked uploads, signed URLs, and MIME-aware processing.

Accomplishments that we're proud of

✅ Full-stack from frontend-only: Shipped Node.js backend, WebSocket streaming, cloud infra as my first real backend project

✅ Buttery-smooth live audio: Sub-second Gemini Live API feels like actual pair programming

✅ 3D avatar that *talks*: Lip-syncs TTS audio in real-time – technical showstopper

✅ Production-grade infra: Terraform + Cloud Build deploys to GCP with zero manual steps

✅ Multi-turn reasoning: Agent remembers conversation context, your codebase, and analysis mode (UI vs perf)

✅ Monaco diff editor: AI patches show as professional unified diffs you can actually apply

What we learned

  • AI accelerates learning curves: Backend/DevOps that would've taken months took 1 week with Gemini help
  • Voice > text for debugging: Natural language + tone makes technical explanations 10x clearer
  • Multimodal > siloed analysis: Screenshot + trace + code together = accurate root cause diagnosis
  • Live audio UX matters: Sub-second latency = delight, 2+ seconds = frustration
  • Infra as code pays off: Terraform/Cloud Build made iteration 10x faster than manual deploys

What's next for FixTrace - Multi-agent frontend debugging platform

  1. Multi-agent swarm: Separate agents for UI, perf, accessibility, security – vote on best fixes
  2. VS Code extension: Debug straight from your editor with live voice
  3. Team collaboration: Shared sessions where designers/devs talk to the same agent
  4. PR automation: Auto-generate pull requests with AI patches + test coverage
  5. Framework translation: "Convert this Angular component to React" with benchmarks
  6. Mobile debugging: React Native/Flutter/iOS screenshot analysis pipeline

The dream: Every frontend team gets a 24/7 debugging agent that speaks their language and ships their fixes. 🚀

Built for #GeminiLiveAgentChallenge using Gemini 2.5 Flash, Live API, TTS, Imagen 4 + Google Cloud Run/Storage/Firebase

Built With

Share this project:

Updates

Submission history