Resonance Switch — Agent Recovery Lab

Inspiration

AI agents can continue using a strategy even after an unexpected environmental change has made it ineffective.

I wanted to make this behavior visible in a small, reproducible developer tool. Resonance Switch focuses on one practical question:

Can an agent recognize when continued progress is no longer working, recover, and resume productive behavior?

What it does

Resonance Switch compares three controllers under the same scenario and seed:

• Always Advance
• Fixed Recovery Rule
• Resonance Switch

The controllers operate with partial observations and choose both a movement action and a public mode: ADVANCE or RECOVER.

Two built-in scenarios are included:

• Normal Route, where the original route remains usable.
• Hidden Blockage, where an unexpected obstacle appears after execution begins.

The simulator records:

• success;
• steps to goal;
• recovery latency;
• repeated ineffective actions;
• mode switches.

How I built it

The project is a deterministic Python command-line tool with a shared interface for all three controllers.

Its public workflow is:

partial observation
→ controller decision
→ ADVANCE or RECOVER
→ environment transition
→ recovery metrics
## Challenges we ran into

The main challenge was creating a fair comparison under partial observation. All three controllers had to receive the same scenario, seed, and public interface without access to future environmental changes.

A second challenge was keeping the project reproducible and easy to inspect. The simulator, metrics, scenarios, tests, and benchmark evidence therefore use deterministic execution and clearly documented limitations.

The Scenario Compiler also required strict validation so that generated scenarios could be tested safely without executing free-form model output.

## Accomplishments that we're proud of

- Built a complete, judge-testable Python developer tool.
- Implemented three controllers using the same public interface.
- Made `ADVANCE` and `RECOVER` behavior visible in traces.
- Created two deterministic partially observed scenarios.
- Recorded a 120-run benchmark using the same seeds for every controller.
- Verified that two deterministic benchmark executions produced identical results.
- Added a natural-language Scenario Compiler with strict local validation.
- Completed 67 passing unit tests.
- Produced a public demo video and a concise cross-platform judge quick start.

## What we learned

Recovery should be evaluated separately from ordinary task success. An agent may eventually succeed while still wasting many steps repeating ineffective behavior after an environmental change.

We also learned that a small benchmark can be useful when its scope, comparison conditions, reproducibility, and limitations are explicit.

Finally, model-generated scenarios should never be executed directly. Structured output and local validation are necessary before a generated scenario enters the simulator.

## What's next for Resonance Switch — Agent Recovery Lab

The next step is to test the same recovery interface with additional environmental changes and external agent adapters.

Potential extensions include:

- new blockage and route-change scenarios;
- tool or workflow failures;
- configurable controller adapters;
- larger but still reproducible evaluation suites;
- integration into agent-development and regression-testing workflows.

The goal is to preserve the same simple question across these extensions: when the current behavior stops working, does the agent recognize the change and recover?

Built With

• agent-benchmarking
• agent-evaluation
• ai-agents
• autonomous-agents
• codex
• command-line-interface
• developer-tools
• gpt-5
• openai-api
• partial-observability
• python
• reproducibility
• simulation
• structured-outputs