CortexLens

AI agents break silently in production. CortexLens finds the exact token count where your agent forgets its rules, which module failed, and why — before your users discover it.

Comment

Inspiration AI agents are being deployed everywhere but nobody has a way to know when they break. We kept asking — what happens to an agent's rules after 50,000 tokens of conversation? Nobody had an answer. CortexLens was built to find it.

What it does CortexLens stress-tests AI agents by flooding them with increasing context load until they break. It identifies the exact token count where the agent fails, which rule it forgot, and which module caused it — then fires a real-time Slack alert with the full failure report.

How we built it We built two systems. PRSentry — a Stripe customer support agent powered by Anthropic Claude with strict rules it must never break. And CortexLens — a chaos engine that floods PRSentry with noise until it breaks. Composio handles Slack alerts, TrueFoundry tracks every token and model call, and ClickHouse stores every test run for pattern analysis. Frontend built in Next.js with a live split-screen showing both systems simultaneously.

Challenges we ran into Making the demo feel real without wasting tokens on every test run. We solved this with a deterministic simulation engine that mirrors exactly how real context degradation behaves — with a Live Mode toggle for real Claude calls when needed. Getting all four sponsor integrations working end to end in one day was the other big challenge.

Accomplishments that we're proud of The Slack alert firing live during the demo with the exact token count, broken rule, and failed module. It stops feeling like a dashboard and starts feeling like a real production monitoring system. Built completely solo in one day.

What we learned AI agents don't fail dramatically — they fail quietly. Small rule violations that compound over time. The scariest failures aren't crashes, they're the $800 refunds that go through silently at 3am with nobody watching.

What's next for CortexLens Connect to any agent via API — not just PRSentry. Auto-generate the stress test scenarios from the agent's own system prompt. Add a memory anchoring feature that automatically reinforces rules at configurable token intervals. And a dashboard that tracks agent reliability across deployments over time.

Built With

  • a-stripe-customer-support-agent-built-on-anthropic-claude
  • and-which-module-caused-it.-demo:-prsentry
  • anthropic
  • clickhouse
  • composi
  • gets-stress-tested-live.-at-48
  • truefoundry
  • which-rule-it-forgot
Share this project:

Updates