casebook-copilot

The model reasons. The casebook survives. Durable incident memory for on-call teams — wrong turns stay visible, and handoffs transfer evidence, not inherited conclusions.

Comment

Casebook Copilot

The model reasons. The casebook survives.

Deployed and live: a published end-to-end run with 34/34 assertions passing · 12 tables · 26 API endpoints · 32 production XanoScript bodies.

Try the live product · View the public repository

The incident ticket is a dead text box

AI chat can help solve an incident, but the incident outlives the chat. It crosses a shift change, a week, or three months — and lands on the next engineer, or your future self, with no reliable memory of what was observed, what was already tried, or why yesterday's diagnosis changed.

Casebook Copilot is durable incident memory for that gap. It preserves the investigation as it happens: byte-exact evidence with provenance, sourced web evidence, a compact current Brief, ruled-out paths, corrections, unresolved questions, a human-readable Case Record, and an append-only audit history.

When work changes hands, the split is deliberate: the person inherits the published investigation record; a fresh AI inherits the evidence — not the previous AI's conclusions.

What makes it different

I started from two frustrations with AI chat during incidents: every answer was long, and nothing held the state of the investigation between answers. Casebook resolves both with one design rather than two features. Because the evidence, the corrections and the ledger are kept in full, an answer can afford to be short — it only has to say what changed this turn. And because the Brief and the Case Record are derived from that record and never fed back into the model, they can orient a human without becoming the model's memory. Chat has to be verbose because the chat is its only record.

Most AI incident tools make the conversation easier to summarize. Casebook asks a different question: what should still be trustworthy after the model changes its mind?

Ordinary AI chat / ticket Casebook
Evidence and interpretation flatten into one transcript Evidence is stored separately from model interpretation
Wrong hypotheses disappear into scrollback Wrong turns and later corrections remain visible
A summary reconstructs the investigation afterward Brief history is recorded while the investigation happens
The next engineer reconstructs context from the thread They receive a deliberately published handover packet
A new AI may inherit the previous AI's framing A fresh-eyes fork receives original evidence, not predecessor reasoning

The demo: the investigation was wrong before it was right

The first incident in the video is real. My own cloud server stopped accepting SSH, and I formed a plausible hypothesis about which firewall was responsible.

It was wrong. New evidence overturned it, SSH was restored, and one question still remained unresolved.

Any AI chat can change its answer when new evidence arrives. The harder problem is preserving what the original evidence actually said, which hypothesis was considered, when it was ruled out, why the investigation changed direction, and what remained unknown when the incident ended. Casebook keeps that history instead of rewriting the investigation into a clean final narrative.

The final minute takes the same idea across people. An investigator publishes a handover packet; another engineer claims it and chooses Take over with fresh eyes. A new case is seeded from the packet's verified evidence — the previous model's conclusions do not come with it. In the demo, the first investigation of an intermittent Kubernetes Unauthorized incident had anchored on certificates. The fresh investigation starts from the same evidence, looks elsewhere, and finds the skewed clock.

The human gets continuity. The AI gets a fresh start on the evidence.

Read against the judging criteria

Progress. This is a deployed product, not a mockup. The complete investigation loop is live — evidence → AI answer → versioned Brief → audit history → Case Record — and the handoff layer adds publish packet → claim → comments/redaction → fresh-eyes fork → resolve. The ticket workflow has a published 34-assertion live end-to-end run: 34 pass, 0 fail, covering happy paths and adversarial cases — concurrent duplicate submissions, stale-version rejection, competing claims, torn-packet rejection, idempotent retries, unauthorized forks, checksum verification, and redaction. The exact script and complete output are in the repository.

Concept. Plain AI chat keeps answers. Casebook keeps why the answer changed. The model is allowed to reinterpret an incident — the original evidence is not silently rewritten with it. The Brief helps the investigator see where the case stands, but is deliberately not fed back to the model as authoritative memory. During handoff the separation becomes stricter: the next human can inspect the published record, while a fresh AI receives only the evidence.

Feasibility. Casebook does not need to replace Jira, PagerDuty, or ServiceNow. It targets the weak layer inside them: the chronological worklog that teams later have to reconstruct into investigation state. The natural users are SRE, platform, and MSP teams whose incidents outlive one person's shift. The first effects worth measuring are concrete: repeated checks avoided after handoff, time-to-context for the next investigator, and MTTR on incidents spanning shifts.

I built it twice

The first version lost to plain chat.

My original design gave the model a much richer persistent state machine: hypotheses, uncertainties, attempted checks, resolution states, and other structured judgments were carried into later reasoning. It sounded safer than ordinary chat. In testing, it was not. A bad semantic classification could become persistent state — and then the premise for every later turn. In another failure, bookkeeping broke after a useful answer had already been generated, and the user lost the answer too.

Those failures changed the architecture. I stopped trying to turn the model's entire judgment process into durable truth and rebuilt around a much smaller separation: evidence stays evidence, model reasoning can change, human summaries help orientation, and bookkeeping cannot delete a useful answer. The previous implementation was not ported — the hackathon version was rebuilt in XanoScript during the submission window.

What Xano actually does

Xano is not just the database behind Casebook. It is the system of record and application orchestrator. Across the core and ticket layer, the deployed build contains 12 tables, 26 API endpoints, 6 reusable production function stacks, and 32 production .xs bodies, handling:

  • authentication, ownership, evidence persistence, and transaction boundaries
  • asynchronous workers and the pending → finalized | abandoned lifecycle
  • Brief and Record persistence, web-evidence ingestion, and the append-only ledger
  • ticket publishing and claiming, idempotency, optimistic concurrency, handover snapshots, redaction, and fresh-eyes case linking

The browser is intentionally thin: submit work, render state, and poll. OpenAI handles the reasoning and recorder roles. SerpApi supplies real-time web evidence with source URLs.

How the record stays trustworthy

Casebook separates three human-facing artifacts because they solve different problems:

  • Brief — where are we now? A compact five-field orientation layer. It helps the human, but does not become authoritative model memory.
  • Case Record — what happened? A readable account of established observations, ruled-out paths, actions, corrections, and unresolved questions.
  • Ledger — what was recorded at the time? An append-only history containing Brief versions and meaningful model/system events.

A later summary can reconstruct what an investigation probably meant earlier. It cannot create a turn-two artifact that genuinely existed before turn six. The answer and recorder paths are separated too: if the recorder fails after a valid answer has been stored, the answer remains available and the Brief degrades instead.

The same principle applies to handoff. A publisher deliberately creates a packet containing the Record, the current Brief, and optionally raw evidence; included evidence carries server-computed SHA-256 checksums. For a fresh-eyes takeover, the evidence is re-verified, a new case is created, the original evidence is seeded into it, and predecessor reasoning stays behind.

This is the lesson from the failed first architecture turned into a product boundary: prompts can influence behavior — guarantees belong in code and system boundaries.

Verification, not just a happy-path demo

Beyond the 34-assertion E2E run, I ran a small answer-quality smoke test: two incidents, Casebook versus plain chat through the same model, five runs per condition. Symptom linking was identical, and cause identification slightly favored the control. The clearest directional difference was discrimination between plausible alternatives: 10/10 Casebook answers received full marks versus 6/10 control answers.

The experiment is small and the grader was itself a language model, so I do not call it proof — it is a smoke test. The scores and methodology are published in the repository.

Known boundaries

The current system has limits, and I would rather state them than hide them.

  • Evidence does not become objectively true merely because Casebook stores it. What Casebook preserves is the authoritative record of what was observed or submitted, without allowing later model interpretation to rewrite it.
  • Handover packets are publisher-attested. The ticket layer verifies packet integrity, but cannot independently prove that the publisher represented the source case honestly.
  • The live instance currently operates as one implicit organization; full tenant isolation is a next step.
  • Explicit recovery can race with a still-running worker inside a documented unfenced window. A worker lease is the known fix.
  • Append-only evidence creates a long-horizon context-growth problem. I intentionally did not ship a compression layer whose failure modes I had not measured.

These are product boundaries, not solved problems disguised as claims.

Build story

Replaced: the incident record/worklog inside systems such as PagerDuty, Jira Service Management, and ServiceNow. A chronological ticket preserves messages, not investigation state — evidence, wrong hypotheses, corrections, ruled-out branches, and unresolved questions flatten into the same timeline, and every handoff becomes reconstruction.

Built with: Xano for auth, data, APIs, asynchronous execution, lifecycle management, concurrency, and auditability; OpenAI for investigation reasoning and record generation; SerpApi for sourced web evidence.

AI tools used while building: Claude Code for application and Xano Metadata API iteration, and Xano's agent for implementing function stacks from written specifications.

Approximate build time: 50–60 hours during the submission window.

What AI + Xano accelerated: I did not spend the hackathon rebuilding authentication, transaction handling, background-worker infrastructure, optimistic concurrency, and an audit database from scratch. That left the build window for the question I actually wanted to investigate: what should the model be allowed to judge, what must remain evidence, and what should survive after the model changes its mind?

What's next

The original next step was shared cases. That shipped during the hackathon — but as published packets and fresh-eyes forks, not shared chats. Next are organization boundaries, scoped and expiring handover links, and longer-horizon experiments around what, if anything, deserves to re-enter model context after a handoff. The rule stays the same: measure first, then decide what deserves to become architecture.

A four-minute demo can show one investigation and one handoff. It cannot show the hundredth case arriving beside the previous ninety-nine. One case is a handoff; a hundred cases are an organization's incident memory.

The demo shows the unit. The product is the accumulation.

Try the live product · View the public repository

Built With

Share this project:

Updates

Submission history