Inspiration
LLM applications rely on external providers that can fail unexpectedly. Rate limits, timeouts, overloaded models, unavailable models, or failed tool-calling continuations can make an otherwise correct application unusable.
The idea for this project emerged while I was building llm-api-adapter. I wanted applications to switch between OpenAI, Anthropic, and Google without rewriting provider-specific integration code.
Once that provider-neutral interface existed, the next question became natural: what should happen when the selected provider is unavailable, rate-limited, or fails during a session? That question led to llm-api-resilience.
For nearly two years, I have been developing llm-api-adapter, a lightweight, unified adapter for OpenAI, Anthropic, and Google. At the time of submission, it had been downloaded 8,542 times from PyPI and had 14 GitHub stars, despite being promoted only through technical posts on LinkedIn and DEV.to and without paid marketing.
This organic adoption showed me that developers value a simple, provider-neutral LLM foundation. llm-api-resilience is the natural next step: while llm-api-adapter standardizes how applications communicate with different providers, this project standardizes what happens when a provider fails.
What I Built
llm-api-resilience is a Python resilience layer for multi-provider LLM applications. It manages ordered routes across models and providers, retries configured transient failures, and switches to another route while preserving a consistent application interface.
Each route uses its own adapter instance and can define a provider, model, timeout, retry policy, and optional prompt profile. The original adapter remains unchanged.
For tool-calling sessions, the application creates a checkpoint before the first tool round and records tool executions in a journal. If a continuation fails, the session can rebuild its state from the checkpoint and replay stored tool results through another model or provider. This allows the application to avoid blindly executing the same side effect twice.
The system also records attempt metadata and supports circuit breaking for unstable routes.
Development Workflow
I use Notion as an external planning and reasoning layer for the project. I first create an implementation plan and then break each implementation item into a detailed commit plan. This keeps changes small, reviewable, and easy to verify. It also helps me check whether Codex understands the intended architecture before a large amount of code is written.
Instead of maintaining a large documentation hierarchy inside the repository, I keep architecture decisions and detailed implementation plans in Notion. For each task, I provide Codex with the relevant project context and only the focused notes needed for that part of the implementation. This keeps the workflow clear and reduces unnecessary context.
Notion also allows me to review architecture decisions, plan features, and investigate bugs from different devices.
I used Codex and GPT-5.6 to inspect the architecture, implement planned changes, generate failure-path tests, and review whether the resulting code matched the implementation plan. I also used Claude Code as a comparison point for evaluating different AI-assisted development workflows.
Log in or sign up for Devpost to join the conversation.