AgentGuard CI

Inspiration

AI agents can now repair code, operate tools, and trigger enterprise workflows, but a successful-looking run is not enough. An agent can still weaken tests, make unrelated diffs, ignore approval boundaries, expose secrets, or invent evidence. AgentGuard CI was built to answer the release question that matters for enterprise automation: is this agent action safe enough to promote, does it need human review, or should it be blocked?

What it does

AgentGuard CI is a reliability firewall for AI agents. It runs repeatable scenarios, captures tool calls and evidence, scores the run across reliability gates, and produces a clear decision: promote, review, or block.

For the UiPath AgentHack Track 3 submission, AgentGuard maps agent reliability scenarios into a Test Cloud-style governance model. Each run produces JSON, Markdown, JUnit, and Test Cloud-friendly evidence that can be attached to a governed test case and reviewed by a human approver.

How it uses UiPath

The project uses UiPath Test Cloud as the target governance layer for repeatable agent reliability cases. It also includes a Studio Web / Labs validation path: the hackathon tenant contains an AgentGuard review agent published as version 1.0.2 with AutoDeploy-Solution active. The hosted review agent evaluates the real release evidence and returns a decision, risk score, evidence mapping, required fixes, and JSON summary.

How I built it

The implementation is a TypeScript monorepo with a React dashboard, Express API, Vitest tests, a scenario runner, a code-fixing agent adapter, and live-local enterprise agent traces. The command-backed suite covers 24 code-agent failure modes. The broader adapter suite covers enterprise surfaces such as browser/RPA, data analysis, customer support, workflow automation, document compliance, finance, HR, CRM, SOC, knowledge retrieval, and multi-agent coordination.

What is proven

The public repository now has a one-command judge path:

npm install
npm run submission:check

That command builds the workspace, regenerates AgentGuard evidence, runs the agent adapter suite, and verifies the UiPath submission assets. The latest evidence shows 204 local tests passing, 24 code-agent failure modes, 106/131 risk points stopped, 5 critical findings, and hosted UiPath Labs validation returning Needs Human Review with a 78/100 risk score.

Challenges

The hardest part was treating agent behavior itself as the test target. A normal CI pass only tells us the code ended green. AgentGuard also checks whether the agent preserved tests, stayed within file scope, produced evidence, respected human approval, and avoided unsafe shortcuts.

What I learned

Agentic automation needs release governance, not just model evaluation. The useful question is not whether the agent sounded confident, but whether its action is traceable, bounded, reproducible, and safe enough to approve.

What's next

Next I would connect the runner more deeply to UiPath Orchestrator and Test Cloud execution records, attach evidence automatically to each governed case, and extend the same promote / review / block contract to more production RPA, data, document, and support-agent workflows.

Built With

• express.js
• junit
• markdown
• node.js
• playwright
• react
• rpa
• typescript
• uipath-test-cloud
• vitest