MAYDAY - RECON

We break your finance agent on purpose - then teach it not to break the same way twice. MAYDAY RECON: because production is a terrible place to discover edge cases.

Comment

MAYDAY RECON

Inspiration

As AI agents move from answering questions to taking actions, reliability becomes much more important than simply getting a correct-looking response. We were particularly interested in finance, where an agent making one seemingly small mistake—such as reconciling the wrong transaction or retrying a successful operation—can create a real financial problem.

That led us to a simple question:

What if, instead of waiting for an agent to fail in production, we built a system whose job was to deliberately make it fail first?

This idea became MAYDAY RECON, an autonomous reliability engineering system for bank-reconciliation agents. We chose bank reconciliation because it provides something extremely valuable for evaluating agents: a deterministic financial environment where we can compare what the agent said with what actually happened to the underlying financial state.

What We Built

We built a simulated finance environment containing bank transactions, general-ledger entries, reconciliations, exceptions, accounting policies, and audit events.

Inside this environment, ReconBot acts as the finance agent. Its job is simple: reconcile bank transactions against the general ledger, automatically resolve safe matches, and escalate ambiguous or unsafe cases for human review.

MAYDAY sits around ReconBot and continuously tests it with adversarial scenarios.

For example, MAYDAY can create situations where:

  • multiple GL entries look equally valid;
  • the GL contains stale information;
  • an API returns only a partial result;
  • a transaction crosses an accounting-period boundary;
  • a reconciliation has already been completed;
  • a mutation succeeds but the API response times out;
  • a reconciliation succeeds but the audit event fails;
  • accounting policies provide contradictory evidence.

The system then observes the actual financial state and determines whether the agent behaved correctly.

The key loop is:

ReconBot
   ↓
MAYDAY Attack
   ↓
Finance Environment
   ↓
Trace
   ↓
Deterministic Evaluation
   ↓
Failure Analysis
   ↓
Regression
   ↓
Intervention
   ↓
New Agent Version
   ↓
Holdout Test
   ↓
Reliability Memory
   ↓
Harder Attacks

A failure is therefore not just recorded as a failed test. MAYDAY turns it into a permanent regression, searches for a possible intervention, tests that intervention against previous failures, and measures whether the new agent actually improved.

How We Built It

We deliberately kept the system focused on one workflow: bank reconciliation rather than trying to build a generic autonomous finance platform.

The architecture consists of several components:

1. CFO Simulator

We created a deterministic SQLite-based finance environment containing:

bank_transactions
ledger_entries
reconciliations
reconciliation_items
exceptions
policies
audit_events
agent_runs

The environment can be reset and replayed so that the same attack produces the same starting conditions and can be evaluated repeatedly.

2. ReconBot

ReconBot is a tool-using agent with access to finance operations such as:

get_bank_transactions()
get_gl_entries()
get_reconciliation_status()
search_accounting_policy()
create_reconciliation()
create_exception()
write_audit_event()

Its decisions are reduced to three outcomes:

$$ \text{AUTO},\quad \text{REVIEW},\quad \text{BLOCK} $$

This makes its behavior easier to evaluate and gives MAYDAY clear safety boundaries.

3. Tool Gateway

ReconBot never communicates directly with the simulator. Every tool call passes through a gateway that records the request, response, latency, errors, attack state, and financial state before and after the operation.

This allowed us to reproduce failures such as:

create_reconciliation()
        ↓
Database commit succeeds
        ↓
API response times out
        ↓
Agent assumes failure
        ↓
Agent retries
        ↓
Duplicate reconciliation

This became one of our most important adversarial scenarios.

4. Attack Engine

We created a YAML-based attack system so that adversarial scenarios could be described as data rather than hard-coded into the application.

Each attack specifies its setup, expected behavior, forbidden behavior, severity, and replay information.

5. Deterministic Evaluator

One of our most important design decisions was not to use an LLM to decide whether a financial mutation was correct.

Instead, MAYDAY compares the expected financial state with the actual state and checks properties such as:

  • whether the correct transaction was selected;
  • whether ambiguity was detected;
  • whether a duplicate reconciliation was created;
  • whether an already-reconciled item was modified;
  • whether policy was violated;
  • whether an audit event was written;
  • whether a partial mutation occurred;
  • whether the agent recovered correctly.

The LLM is used later for tasks where semantic reasoning is useful, such as explaining the root cause of a failure.

6. Failure → Regression → Intervention

When MAYDAY detects a failure, a failure analyzer identifies the likely root cause and supporting evidence.

The failure is then converted into a deterministic regression test.

MAYDAY can subsequently test bounded interventions such as:

Prompt changes
Workflow guards
Tool interventions
Policy rules
Memory rules

Each intervention is tested against both the original failure and the existing regression suite before it is accepted.

7. Reliability Memory

MAYDAY stores historical failure patterns, agent weaknesses, attack success rates, successful mitigations, and regression tests.

This allows future testing to focus on areas where the agent has historically been weak rather than simply running a fixed collection of tests forever.

What We Learned

The biggest lesson was that agent evaluation cannot stop at the final response.

An agent can produce a convincing explanation while still leaving the financial system in an unsafe state. For tool-using agents, the environment itself has to be part of the evaluation.

We also learned that reliability is not simply a single accuracy number. For our system, an agent must be evaluated across correctness, financial safety, recovery, policy compliance, auditability, latency, and cost.

Another important lesson was the value of regression as memory. A failure is useful only if the system remembers it. By automatically converting failures into replayable tests, every mistake becomes part of the agent's future defense.

Most importantly, we learned to distinguish memorization from learning. We therefore designed separate training and holdout scenarios. MAYDAY is allowed to learn from the training failures, but the holdout scenarios remain hidden. If the improved agent performs better on unseen failures, we have evidence that the system learned a broader reliability pattern rather than simply memorizing individual attacks.

Challenges

Making failures deterministic

Real-world failures are often unpredictable. For a hackathon project, however, we needed failures that could be reproduced exactly.

We solved this by creating a deterministic finance simulator, resettable environments, replayable attacks, and state snapshots.

Evaluating financial actions safely

A traditional LLM evaluator could easily decide that an agent's explanation "sounds correct" even when the database was wrong.

We therefore separated truth from explanation:

Deterministic evaluator → What happened?
LLM analyzer            → Why did it happen?

This separation became one of the core principles of MAYDAY.

Designing useful attacks

It was tempting to generate hundreds of random failures, but random noise is not useful if it doesn't represent meaningful financial risk.

We instead focused on a small number of high-value attack classes involving ambiguity, stale data, state conflicts, transaction failures, policy conflicts, and partial operations.

Proving that the system actually improves

Showing that an agent passes more tests after a prompt change isn't enough.

We needed to demonstrate that MAYDAY could:

$$ \text{Discover} \rightarrow \text{Explain} \rightarrow \text{Regress} \rightarrow \text{Repair} \rightarrow \text{Retest} \rightarrow \text{Generalize} $$

That is why the holdout benchmark became such an important part of the project.

The Result

MAYDAY RECON changes the way we think about agent reliability.

Instead of:

Build agent
     ↓
Test agent
     ↓
Deploy agent

we wanted:

Build agent
     ↓
Attack agent
     ↓
Find failure
     ↓
Understand failure
     ↓
Remember failure
     ↓
Repair agent
     ↓
Test repair
     ↓
Attack again

Our goal was never to build a finance agent that claims to be reliable.

We wanted to build a system that continuously tries to prove where the agent is not reliable yet.

We didn't build a reconciliation agent and hope it was safe. We built a system whose job is to continuously prove where it isn't.

Built With

Share this project:

Updates

Submission history