TestForge AI

Natural language to executable UiPath test suites. Claude generates, Test Cloud executes, Claude improves failures. Zero manual test writing.

Comment

Inspiration

QA engineers at enterprises spend 40-60% of their time writing and maintaining test cases for UiPath automations. When a workflow changes, test suites break and someone manually updates each case. TestForge AI eliminates this loop entirely: natural language in, passing tests out.

What it does

TestForge AI is a three-stage AI pipeline built on Claude and UiPath Test Cloud:

  1. Generate -- Describe your UiPath workflow in plain English. Claude generates a comprehensive test suite (5-8 cases) covering happy path, error conditions, edge cases, and boundary values.

  2. Execute -- The test suite is uploaded to UiPath Test Cloud via REST API and executed automatically.

  3. Improve -- Claude analyzes failures, identifies root causes, rewrites failing test cases, and triggers a re-run. The loop continues until pass rate exceeds 95%.

The recursive improvement loop is the key differentiator: most test generation tools stop at generation. TestForge closes the loop.

How we built it

  • Claude claude-sonnet-4-6 via Anthropic Messages API for test case generation and result analysis
  • Claude Code as a coding agent for the improvement loop -- reads failure logs, generates improved test cases, restages execution
  • UiPath Test Cloud API (Orchestrator REST API) for test set creation, execution, and results polling
  • FastAPI REST backend
  • Python 3.11 throughout

The Claude integration uses structured JSON output enforced via system prompt to ensure every generated test case is immediately executable.

Challenges we ran into

Getting Claude to output consistently structured JSON across diverse workflow descriptions was the main challenge. Solution: include explicit JSON schema in the system prompt with example values, strip markdown code fences on the client side before parsing.

The improvement loop required careful design to avoid infinite cycles -- a 95% pass rate threshold and max-iterations cap prevent runaway re-execution.

Accomplishments we're proud of

  • End-to-end pipeline from natural language to Test Cloud results in under 60 seconds
  • Improvement loop reliably takes failing test suites from 70-80% to 95%+ pass rate in 2-3 iterations
  • Claude Code integration enables genuine agentic behavior: the system reads its own outputs, identifies failures, and autonomously rewrites the weak cases

What we learned

Claude excels at structured output when the schema is specified precisely in the system prompt. Vague instructions produce markdown; explicit JSON schemas produce exactly what you need.

What's next for TestForge AI

  • UiPath Studio integration: generate test cases directly from XAML workflow files
  • Slack/Teams notifications when the improvement loop completes
  • Test case versioning: track how tests evolve across workflow iterations
  • Multi-environment support: run the same suite against dev/staging/prod sequentially

Built With

  • anthropic
  • claude-code
  • fastapi
  • json
  • python
  • rest-api
  • uipath
Share this project:

Updates