• 

  POST /adjudicate — a claim billed as covered under the structured policy

• 

  Response: denied, citing the free-text exclusion clause, with full audit trail

claims-triage-agent

An LLM agent that adjudicates health-insurance claims: it reads a claim, calls tools to check things it cannot safely guess (coverage rules, claim history, cost-sharing math, free-text policy clauses), and returns a structured, justified, fully audited decision.

All data (patients, policies, claims, policy documents) is synthetic and invented. No real patient data, real payer data, or proprietary code is used anywhere here.

Why This Project

I built this as a public, from-scratch demonstration of what I consider core to AI engineering: an LLM agent that reads and reasons over documents, with reliability and auditability treated as first-class requirements rather than an afterthought.

What It Does

The agent never guesses. It calls tools, retrieves and cites the exact policy clause behind every decision, and forces human review whenever its own retrieval doesn't back up the call. One test case proves this concretely: a claim where the structured data says a procedure is covered, but the free-text policy clause says it's excluded — the agent only reaches the correct decision by retrieving and citing that clause.

How I Built It

The agent is implemented twice, deliberately, sharing the same tools and reliability logic:

• A hand-rolled tool-calling loop against a plain LLMClient protocol, zero third-party dependencies.
• A LangGraph StateGraph with real nodes, conditional edges, a checkpointer, and a genuine interrupt() call for Human-in-the-Loop review.

The retrieval layer is a from-scratch ~50-line Okapi BM25 implementation, kept dependency-free on purpose. The API is FastAPI, with Docker and a GitHub Actions CI pipeline (ruff, mypy, pytest — 32 tests, all passing).

Reliability Guardrails

Two things are enforced deterministically, not left to the model's judgement:

1. Tool-call budget — if the model hasn't reached a decision within a budget of tool calls, the run is force-flagged for review.
2. Ungrounded decisions — if a policy-document search comes back empty and the final decision doesn't cite any passage, the decision is overridden to flagged_for_review rather than trusting the model's confidence.

Challenges I Ran Into

Making sure the RAG path was actually load-bearing rather than decorative — proving it required a test case where the structured data and the free-text clause disagree, so the correct answer is only reachable through retrieval.

What I Learned

Building the same agent twice — once by hand, once with LangGraph — taught me both what a tool-calling loop actually does underneath, and how to work with the orchestration framework a production team would use day to day.

What's Next

A hybrid lexical + embedding retriever (the current BM25 is lexical-only and misses semantically-related but differently-worded clauses), and a persistent checkpointer (Postgres) so a paused Human-in-the-Loop review survives a process restart.

Built With

• docker
• fastapi
• github-actions
• langchain
• langgraph
• llm-agents
• pydantic
• pytest
• python
• rag