AutoUX

Inspiration

Valuable UX feedback is constantly lost in the noise of software development. We noticed that users often complain about a "confusing button" or a "clunky workflow" deep inside comment threads for completely unrelated backend bug reports. Product Managers and Designers are forced to spend hours manually reading noisy threads, extracting that feedback, manually labeling tickets, and writing up new UX-specific issues. We wanted to eliminate this friction entirely and transform Product Managers from manual triagers to reviewers by automating the entire feedback extraction pipeline.

What it does

AutoUX is an autonomous UX Researcher and Triage Agent built on the GitLab Duo Agent Platform. When a user triggers the flow by commenting @ai-autoux-flow... on an issue, the agent executes a complex, multi-tool workflow:

1. Pre-Filters the Noise: Uses GitLab's native search API to find recent issues containing UX-related keywords.
2. Auto-Labels: Automatically applies a UX-Feedback label to source issues, marking them as triaged.
3. Spins up Actionable Tickets: For critical usability issues, it generates a dedicated [UX Debt] ticket for the design team, complete with a direct blockquote of the original user's complaint for exact context.
4. Updates a Master Report: Instead of spamming the board, it finds and updates a single "📊 Living UX Master Report" with a changelog of its latest findings.

How we built it

We built AutoUX using GitLab's Agent and Flow YAML definitions (flows/flow.yml and agents/agent.yml). We heavily utilized GitLab's built-in toolset—including gitlab_issue_search, list_issue_notes, update_issue, create_issue, and create_issue_note—to give the LLM read/write access to the repository.

To make it context-aware, we injected GitLab's backend {{context}} payload directly into the prompt so the agent could dynamically extract its own Project ID and current issue location without needing human input. We also maintained a parallel agent.yml configuration so users could interact with AutoUX conversationally via the GitLab Duo sidebar.

Challenges we ran into

Building a prototype is easy; building an agent that doesn't break production is hard. We ran into several major "Agentic Bottlenecks":

• Token Exhaustion: A naive agent tries to read the entire issue backlog, wasting thousands of tokens on stack traces and CI/CD logs. We solved this by forcing the agent to use keyword pre-filtering (gitlab_issue_search) and time-boxing its reads to the top 10 results.
• The Continuous Feedback Loop (State Invalidation): If you run an agent twice, it creates duplicate tickets. We solved this by making the agent idempotent—using GitLab labels as its memory so it skips previously processed issues. We then had to build a "Context-Aware Override" so that if a PM explicitly tags the agent in an already labeled issue, it knows to bypass its own filters and extract the new comments.
• Infinite Loops & Spam: The agent initially risked triggering itself by outputting its own @ai- handle. We implemented strict prompt safeguards, hard tool-call quotas (max 3 ticket creations per run), and a "Living Document" architecture to completely eliminate issue tracker spam.
• Strict CI/CD Schemas: GitLab's YAML parser is incredibly strict. We had pipeline failures due to minor indentation mismatches (like the unit_primitives block) which required meticulous formatting to resolve.

Accomplishments that we're proud of

We are most proud of building an enterprise-grade, production-safe AI system. AutoUX isn't just a chatbot wrapper; it's a state-aware background worker. We successfully chained multiple API tools together while implementing strict safeguards (Label-based Idempotency, Search-based Token Efficiency, Loop Prevention, and Traceability) to ensure it behaves predictably in a real-world, high-traffic repository.

What we learned

We learned the intricacies of the new GitLab Duo Agent Platform, specifically how to bridge background Flows with conversational Agents. We gained deep insights into LLM state management (using platform labels as a database) and prompt engineering for complex, multi-step API orchestration. Most importantly, we learned that the biggest challenge in AI engineering isn't getting the LLM to generate text, but constraining it to act safely and efficiently within system limits.

What's next for AutoUX

In the future, we want to expand AutoUX's toolset to automatically assign [UX Debt] tickets to specific designers based on workload, automatically generate Figma placeholder links in the issue description, and perhaps roll up multiple related UX friction points into larger GitLab Epics for quarter-planning.

Built With

• gitlab-agent
• gitlab-flow
• markdown
• yaml