Inspiration
Most AI agents today are static. When an agent fails, a developer must manually inspect the failure, modify prompts or tools, and deploy another version. We wanted to build a system where agents could learn from their own failures and improve automatically. This inspired AgentForce — a self-improving agent engineering platform built around a continuous Execute → Evaluate → Learn → Improve loop.
What it does
AgentForce takes a goal, available tools, and success criteria, then creates and runs a specialized agent. It evaluates the agent's performance, identifies failure patterns, and automatically improves its prompts, tool usage, memory, or orchestration strategy.
It maintains execution history and compares iterations to demonstrate measurable improvements in accuracy, reliability, speed, and cost.
How we built it
We built AgentForce as an agent orchestration and evaluation pipeline using Agent Orchestrator (AO). The system consists of a goal analyzer, agent architect, tool selector, execution engine, evaluator, failure analyzer, improvement engine, and memory layer.
The agent is evaluated against repeatable test cases, allowing us to compare different iterations and quantify improvement.
Challenges we ran into
The biggest challenge was designing a meaningful improvement loop rather than simply regenerating prompts. We needed to distinguish between successful and failed executions, identify the actual cause of failures, and make targeted improvements without increasing cost or complexity unnecessarily.
Another challenge was creating measurable evaluation criteria that could reliably demonstrate progress between agent versions.
Accomplishments that we're proud of
We built a working autonomous improvement loop where an agent can execute, analyze its failures, modify its strategy, and retry with minimal human intervention.
We are particularly proud of making the improvement process measurable and transparent instead of treating agent improvement as a black box.
What we learned
We learned that building reliable agents is not only about choosing a powerful LLM. Tool selection, orchestration, evaluation, memory, failure analysis, and feedback loops are equally important.
We also learned that measurable evaluation is essential for determining whether an agent is genuinely improving.
What's next for AgentForce
Next, we want to expand AgentForce into a general-purpose agent factory capable of creating and improving agents across many domains.
We plan to add stronger long-term memory, automated tool discovery, more sophisticated evaluators, multi-agent collaboration, cost-aware optimization, and continuous production feedback.
Our long-term vision is simple: give AgentForce a goal, and let it build an agent that gets better every time it runs.
Built With
- ao
- llm
- python
