-
-
Output
Inspiration
We built this agent to solve a common and persistent problem in software teams: test suites gradually become flaky, brittle, under-documented, and incomplete. In GitLab-based projects, these issues often remain hidden until they cause failures in development or production. Our goal was to automate test quality auditing and make the results actionable inside the same workflow where code already lives.
What it does
The agent audits a repository’s test suite across five dimensions: flaky tests, brittle tests, missing edge cases, integration coverage gaps, and test documentation quality. It generates a structured Test Health Report and creates fixed test files for the highest-priority issues, all within GitLab.
How we built it
We built the system using Claude AI, GitLab MCP tools, Python, pytest, pytest-mock, freezegun, GitLab CI/CD, and the GitLab Issues API. The agent inspects repository structure, reads tests and source files, reviews pipeline history, and produces fixes in a timestamped output folder.
Challenges we ran into
The main challenges were enforcing reliable tool execution, handling approval-based workflows in GitLab, and ensuring the agent remained project-agnostic while still producing concrete, repo-specific results.
Accomplishments that we're proud of
We created an end-to-end GitLab-native agent that not only identifies test quality problems but also generates practical fixes. The workflow is structured, repeatable, and safe for real repositories.
What we learned
We learned how important tool availability, approval design, and prompt precision are in agent workflows. We also learned how to balance automation with human control in a production-style environment.
Built With
- claude-ai
- freezegun
- gitlab-ci/cd
- gitlab-issues-api
- gitlab-mcp-tools
- pytest
- pytest-mock
- python
