TestCraft Duo

An agentic QA assistant that analyzes new GitLab merge requests, dynamically generates unit and end-to-end integration tests for edge cases, and verifies test coverage automatically.

Comment

Inspiration

Manual regression and edge-case testing remain significant bottlenecks post-code commit. Developers often write unit tests for expected happy paths, but critical failure modes, boundary conditions, and security edge cases escape into continuous integration (CI) pipelines—leading to costly pipeline failures and delayed releases. We were inspired to solve this by creating TestCraft Duo, an autonomous agentic testing workflow built directly on top of the GitLab Duo Agent Platform.

What it does

TestCraft Duo acts as an autonomous QA engineer operating silently post-commit. Whenever a developer opens or updates a merge request (MR), TestCraft Duo:

  1. Analyzes Code Changes: Evaluates the diff against existing codebase patterns and public interfaces.
  2. Generates Target Edge-Case Tests: Automatically crafts high-coverage unit, integration, and boundary test suites using Anthropic Claude models.
  3. Verifies & Auto-Pushes Patch: Executes the new tests inside the GitLab CI/CD runner, verifies execution success, and commits the test patch back to the MR branch—without human intervention.

How we built it

We leveraged the GitLab Duo Agent Platform ecosystem to orchestrate autonomous tool calls and flow executions:

  • GitLab Duo Agent Platform & Flows API: To construct custom agent flows triggering on merge request lifecycle events.
  • Model Context Protocol (MCP) & GitLab GraphQL API: For seamless codebase context retrieval, MR diff inspection, and repository commit actions.
  • Anthropic Claude 3.5 Sonnet / Claude 3 Opus (via GitLab Duo AI Gateway): Powering deep static code analysis and precise test synthesis.
  • GitLab CI/CD Runners: For isolated test execution and validation.

Challenges we faced

  • Context Management: Preventing LLM prompt context bloat on large code diffs. We resolved this by querying structural abstract syntax trees (ASTs) through custom Model Context Protocol (MCP) servers.
  • Loop Prevention: Ensuring that automated commits pushed by TestCraft Duo do not trigger infinite CI/CD test generation loops by implementing smart agent privilege checks and session tags.

What we learned

Building with the GitLab Duo Agent Platform revealed the immense power of agentic workflows over standard chatbots. Shifting from interactive chat to event-driven, post-commit background agents drastically reduces developer context-switching.

What's next for TestCraft Duo

  • Adding automated visual regression testing using headless browser instances in CI.
  • Incorporating SAST/DAST security test case auto-generation for flagged dependencies.

Built With

  • anthropic-claude
  • gitlab-ci-cd
  • gitlab-duo-agent-platform
  • google-cloud
  • graphql-api
  • model-context-protocol-mcp
  • python
  • vue-js
Share this project:

Updates

Submission history