AgentFuzz

Inspiration

AI assistants are becoming capable of doing more than answering questions. They can use tools to schedule meetings, manage reminders, send messages, and control smart-home devices. But what happens when one of those tools fails halfway through a task?

An assistant might continue executing actions, leave the application in an inconsistent state, and confidently tell the user that everything worked. Ordinary happy-path testing can miss these failures.

We built AgentFuzz to investigate this problem: how can developers systematically discover, reproduce, and fix reliability failures in tool-using AI assistants before those failures reach users?

What it does

AgentFuzz is a local-first reliability testing workbench for simulated Alexa+-style AI assistants.

Instead of only testing whether an assistant can complete a normal task, AgentFuzz introduces failures and checks whether the assistant behaves correctly.

Its main capabilities include:

• Stateful assistant simulation: A simulated home assistant interacts with calendar, reminders, messaging, smart-home, and shopping-list tools.
• Fault injection: Scenarios introduce conditions such as scheduling conflicts, timeouts, and missing data.
• Deterministic failure detection: AgentFuzz checks for incorrect or duplicate state changes, false completion claims, and lost state.
• Fuzzing and minimization: It generates and mutates scenarios, identifies failures, and attempts to reduce them to smaller reproducible cases.
• Benchmarking: It compares hand-written tests with random and mutated testing under a controlled execution budget.
• Fix verification: It replays a discovered failure against a corrected agent and checks held-out regression scenarios.

How we built it

AgentFuzz uses Python and FastAPI for its backend, a browser-based interface for the simulator and Reliability Rig, and a locally running Ollama model for agent interactions. Its mock tools maintain application state so the test harness can compare what the assistant claims to have done with what actually happened.

A central example is a calendar conflict. The original agent can encounter a failed meeting update, continue with a dependent action, and incorrectly claim success. AgentFuzz detects the discrepancy and preserves a reproducible failure trace.

We then implemented general tool-error handling so the corrected agent stops dependent actions after a tool failure and reports the problem honestly. The verification workflow replays the original failure and checks five held-out regression scenarios.

The project is designed to run locally, without requiring paid model APIs or cloud deployment. It simulates an Alexa+-style experience; it does not connect to real Alexa+ services or physical Echo devices.

Challenges we faced

One challenge was distinguishing genuine failures from misleading test results. A mutated scenario can leave behind stale expectations even after a fault is removed, creating a false positive. We therefore separated genuine detections from false positives and recorded the limitations of the benchmark.

Another challenge was making failures reproducible. A failure report is far more useful when a developer can replay the exact scenario, inspect the resulting state, apply a fix, and run regression checks.

We also had to keep the fix general rather than hard-code a special response for one calendar scenario. The goal was to improve how the agent handles tool failures across different tasks.

Results and what we learned

In our measured local benchmark, hand-written tests repeatedly detected known regressions, while random and mutated testing uncovered two novel failure signatures. The mutated strategy also produced false positives, demonstrating why automated testing needs careful validation rather than simply counting every detected issue as a bug.

We learned that reliable AI agents require more than good language-model responses. Their tool calls, application state, error handling, and completion claims all need to be tested together.

AgentFuzz is a prototype demonstrating this approach. Its benchmark is small, and broader testing would be needed to establish how well the method generalizes to other agents.

What's next

The next step is to expand the scenario corpus, improve fuzzing and failure minimization, and test the approach against additional agent implementations and more complex multi-step tasks.

Our goal is simple: help developers find failures before users experience them.

Built With

• agentic
• agents
• ai
• css
• engineering
• fastapi
• fault
• fuzz
• html
• injection
• javascript
• language
• large
• models
• ollama
• open
• pytest
• python
• qwen2.5
• reliability
• software
• testing