testforge-agent

Automatically generates unit and integration tests from MR diffs

Comment

Inspiration

Writing tests is one of the most skipped steps in fast-moving teams. Code review tools flag missing coverage, but nobody fixes it. Junior developers and AI-generated code often ship with zero tests. We wanted an agent that doesn’t just suggest — it acts. TestForge is a digital teammate that jumps into the workflow when you mention it in an MR comment and generates tests for you.

What it does

TestForge analyzes merge request diffs, identifies changed source files (Python, JavaScript, TypeScript), and generates unit tests. Mention @ai-testforge-gitlab-ai-hackathon_b6fec3 in an MR comment with the MR URL — it reads the diff, creates test files (pytest for Python, Jest/Vitest for JS/TS), commits them to the branch, and posts a summary. No chatbots. It takes action.

How we built it

  1. Flow definition — YAML flow with an agent component, system prompt for test generation, and toolset (gitlab_api_get, list_merge_request_diffs, get_repository_file, create_file_with_contents, create_merge_request_note)
  2. Prompt engineering — Instructions for parsing MR diffs, identifying testable files, generating tests with happy path, edge cases, and error handling
  3. Mention trigger — Flow runs when the service account is mentioned in an MR comment
  4. Service account — Added to the project so the agent can read files, create commits, and post notes

Challenges we ran into

  • 404 Project Not Found — The platform passed the service account name as the project identifier instead of the MR’s project. We solved it by having users include the full MR URL in the comment, which the agent parses correctly.
  • Flow sync — The flow in the AI Catalog is separate from the repo. We documented the working trigger and kept the flow YAML in the repo for reference.

Accomplishments that we're proud of

  • Built a working agent that reacts to triggers and takes real action (creates files, commits, posts notes)
  • Generated 100+ tests across 6+ files in a single MR run
  • Supports both Python (pytest) and JavaScript/TypeScript (Jest/Vitest)
  • Clear documentation and a reliable trigger command for users

What we learned

-> How to build flows in the GitLab Duo Agent Platform that react to triggers and orchestrate multiple tools -> The importance of project context — and how to work around platform limitations -> Prompt engineering for structured output: test generation with consistent naming, coverage, and style

What's next for testforge-agent

->Fix project context so "Generate tests for this MR" works without the URL (platform or flow update) ->Add support for more languages (Go, Rust, Ruby) ->Integrate with CI to run generated tests and report coverage back to the MR

Built With

  • gitlab-api
  • gitlab-duo-agent-platform
  • jest
  • pytest
  • python
  • vitest
  • yaml
Share this project:

Updates