Ironclad Returns

The AI returns desk that can't be talked out of its policy — jailbreak-proof, deterministic, fully auditable.

Comment

Inspiration

The project's inspiration is base on a 2024 court case where a tribunal ordered an airline to honor a refund policy its support chatbot had invented (Moffatt v. Air Canada). That case names the real problem with customer-facing AI: it's not that models are impolite — it's that the model owns the decision. If language can reach the decision, persuasion can too.

Ironclad Returns inverts that architecture. The AI is allowed to understand you. It is not allowed to decide anything.

What it does

A customer opens a public returns portal (no login) and requests a return in plain language. Under the hood:

  • The LLM has exactly one job: emit structured JSON — extract an order reference, classify a return reason. It never selects a transition or an outcome.
  • A deterministic finite-state machine decides everything: order lookup, delivery-window eligibility, refund math with restocking fees, and a high-value threshold that routes big refunds to human review.
  • Guards run before the model on every turn. Prompt injection ("ignore your instructions…") and social engineering ("the manager said…") are intercepted and routed to a calm policy response — even mid-flow, one step from an approval.
  • Every outcome is computed, auditable, replayable. A decision-audit view shows each session's full trace: slots captured, computations run, guards fired, and the exact transition that ended it.

How we built it

The project was scaffolded with Kubild, a spec-driven platform generator (a pre-existing tool created by the author; generator state as of commit 8769b2e), which provided the base SaaS shell and a generic behavior-engine runtime. Everything that makes this entry Ironclad was built during the submission period with OpenAI Codex (session 019f7200-8a26-7ff2-8337-7158cd00d283): the returns behavior pack and its deterministic computes, the adversarial guard set, the 42-scenario offline eval suite, the public returns portal, the decision-audit trail, and all fixes and polish.

Challenges we ran into

  • A public, anonymous portal on top of an authenticated scaffold: a route-mounting bug returned 401s (the 1 ms rejection time was the tell), and CSRF had to keep working for cookie-less visitors without weakening the authenticated app.
  • Guards that don't overreact: phrase-level patterns plus an explicit false-positive eval — a customer asking "your policy says 30 days, right?" must get an answer, not a security response.
  • Honest conversation design: when a customer replies with a question instead of a reason, the classifier restates the policy and re-asks instead of guessing.

Accomplishments that we're proud of

  • We had Codex attack its own guards — during development it found five bypass phrasings, and live testing surfaced more, including the canonical "ignore your previous instructions." That drove us from brittle keyword matching to a regex matcher covering the whole injection family, backed by a false-positive suite proving real customers still pass. Every attack that got through became a permanent test.
  • 42/42 eval scenarios passed, 20/20 guard tests passed — including first-turn injection, mid-flow injection after order lookup, manager pressure, and chargeback threats. The trust claim isn't asserted; it's tested.
  • A stranger can open the portal, complete a return, fail to jailbreak it, and then an auditor can replay exactly why every outcome happened.

What we learned

  • The reliable pattern for AI in high-stakes workflows: language layer and action layer must be separate. Let the model interpret; let deterministic code decide.
  • Evals are the product's trust artifact. Twenty guard scenarios running in CI is worth more than any promise in a system prompt.
  • Adversarial self-testing works: the model that wrote the guards was also the best tool for finding their holes.

What's next for Ironclad Returns

  • A live public demo URL. Try it live: https://ironclad-returns-production.up.railway.app/returns — attempt a jailbreak from your browser right now.
  • Real order sources (Shopify) and refund execution — the demo computes and explains decisions; production connects them to money movement.
  • The same architecture generalizes to any policy-driven desk: warranty claims, service intake, collections — anywhere a company needs AI that listens but cannot be persuaded.

Built With

Share this project:

Updates