APforge — Agents That Learn From Their Failures
Inspiration
Most AI agents are good at executing instructions, but they are often stuck with a fixed strategy. When a tool fails or produces the wrong information, the agent may simply try again without understanding what went wrong.
We wanted to build something different:
What if an agent could turn its own execution failures into experience and use that experience to improve future tool selection?
That idea became APforge — an autonomous agent engineering system that learns how to use tools more effectively through experience.
What APforge Does
APforge takes a task and a set of available tools and runs an agent against the task.
Instead of treating every execution as independent, APforge creates a continuous learning loop:
USE → OBSERVE → EVALUATE → REFLECT → REMEMBER → IMPROVE
The system:
- Executes a task using available tools.
- Records the complete execution trace.
- Evaluates whether the result is correct.
- Classifies failures and identifies what went wrong.
- Generates reflection and alternative tool-use strategies.
- Stores the experience in persistent memory.
- Updates tool-selection strategy scores.
- Creates a new agent version.
- Uses the learned strategy on future tasks.
The goal isn't simply to make one task succeed.
The goal is to make the agent itself better at solving future tasks.
The Learning Mechanism
APforge maintains persistent experience in SQLite.
For each task type and tool, the system tracks experience such as:
- successful tool usage
- failed tool usage
- failure types
- alternative tools
- learned strategy adjustments
- empirical tool performance
- agent versions
Successful choices are reinforced while failed choices are penalized. Reflection can also boost alternative strategies suggested after failures.
This means the next agent version does not start from zero.
It inherits what the previous agent learned.
Evaluation
We built a deterministic engineering-investigation benchmark containing four task suites:
- CI Build Investigation
- Production Outage Triage
- Native Crash Debugging
- Performance Regression Investigation
Each scenario provides multiple possible instruments, including correct readers, decoys, failing tools, and degraded alternatives.
Held-out benchmark
| Metric | V1 | V2 | V3 |
|---|---|---|---|
| Solve rate | 0% | 40% | 100% |
| Step success | 0% | 83.3% | 100% |
| Total calls | 216 | 96 | 72 |
| Unnecessary calls | 216 | 36 | 0 |
| Failures | 102 | 0 | 0 |
| Total latency | 2943 ms | 582 ms | 454 ms |
| Modeled cost | 327.30 | 96 | 72 |
Unseen-task evaluation
| Metric | V1 | V2 | V3 |
|---|---|---|---|
| Solve rate | 0% | 33.3% | 100% |
| Step success | 0% | 81.8% | 100% |
| Total calls | 264 | 120 | 88 |
| Unnecessary calls | 264 | 48 | 0 |
| Failures | 126 | 0 | 0 |
| Total latency | 3612 ms | 731 ms | 554 ms |
| Modeled cost | 400.80 | 120 | 88 |
These numbers come from actual executions of our deterministic evaluation harness.
The key result is that APforge improved from a naive strategy to a learned strategy while reducing unnecessary tool usage.
Live Learning Replay
The project also includes an interactive learning replay.
The replay intentionally exposes the agent to failures. Instead of hiding those failures, APforge captures them as experience and uses them to update its strategy and memory.
This makes the learning process observable:
Failure → Analysis → Memory → Strategy Update → New Agent Version
The interactive replay runs the actual APforge engine and tool execution path, while the benchmark results provide the quantitative V1 → V2 → V3 evaluation.
How We Built It
The core system is implemented in Python with a modular architecture:
- Agent Engine — executes tasks and coordinates the learning loop
- Tool Registry — manages available tools
- Execution Traces — records every tool interaction
- Evaluation Engine — evaluates task outcomes
- Failure Analysis — classifies execution failures
- Reflection Engine — generates improvement signals
- Persistent Memory — stores tool-use experience in SQLite
- Strategy Engine — converts experience into future tool-selection behavior
- Versioning — tracks agent evolution
- Benchmark Harness — evaluates learning across held-out and unseen tasks
- Web Dashboard — visualizes execution, memory, strategies, and benchmark results
APforge was developed using Agent Orchestrator (AO) throughout the development process, including implementation, testing, evaluation, and debugging.
Challenges
One of the biggest challenges was making the learning process measurable rather than simply claiming that the agent "learned."
We therefore designed an evaluation harness where:
- tasks have explicit success criteria
- tool identity is not exposed through ground truth
- decoy tools can fail
- execution traces are recorded
- learning happens through observed experience
- V1, V2, and V3 can be compared quantitatively
- unseen-task instances can test generalization
Another challenge was making the live demo work reliably in a serverless environment. The production replay was separated from the benchmark artifact so the interactive demo could execute without depending on server-side result-file writes.
We also added tests around memory persistence, strategy updates, agent versioning, benchmark evaluation, API behavior, and the live replay.
What We Learned
The biggest lesson was that agent improvement does not necessarily require retraining a model.
A large part of an agent's behavior comes from:
which tool it chooses, when it chooses it, and what it remembers about previous attempts.
By making those decisions observable and learnable, we can build agents that improve their behavior through experience.
We also learned that evaluation is critical. A learning system needs measurable evidence that its behavior actually changed for the better.
Why APforge Matters
APforge is designed as a foundation for specialized agents that encounter unfamiliar workflows.
The same architecture can be extended to:
- software engineering agents
- SRE and incident-response agents
- research agents
- data-analysis agents
- automation agents
- multi-tool enterprise agents
Instead of manually designing every workflow, an agent can learn better strategies from the outcomes of previous executions.
Final Thought
Traditional agents execute instructions.
APforge learns how to execute them better.
APforge doesn't just execute tasks. It learns how to execute them better.
Log in or sign up for Devpost to join the conversation.