Inspiration

It started with a missed auto-renewal.

A contract rolled into another year because nobody caught the 60-day non-renewal window buried on page 9 of an MSA that had been amended twice. The obligation was clear, written in plain English, and completely invisible to the systems the business actually ran on.

That's when the pattern clicked: contracts are the only business documents where the content is the instruction. They tell you what to do, when, and by whom — yet every organization treats them as static PDFs. So we asked a simple question:

What if a contract wasn't a document, but a program? What if we could compile it?

Contracts have declarations (defined terms), statements (obligations), control flow (if, provided that, subject to), precedence rules, versions and patches (amendments), and undefined behavior (ambiguity). That's a programming language. If contracts are source code, compliance is the runtime — and nobody had built the compiler.

What it does

Our AI agent converts a stack of contract documents — MSA, SOW, Order Forms, Amendments, DPAs, policies — into executable compliance workflows.

The pipeline:

Ingest & hierarchy — classifies each document, links amendments to what they amend, builds a version-aware document graph

Extract obligations — schema-constrained extraction of actor, action, object, modality, trigger, deadline, and condition — with a mandatory source span for every field

Resolve parties & deadlines — maps defined terms → legal entities → internal owners; computes absolute, relative, event-triggered, and recurring dates with business-day and timezone rules

Reason over conditionals — compiles nested conditions into evaluable decision graphs; unknown facts become data-collection tasks

Reconcile conflicts — ordered precedence engine (express clause → document rank → specific over general → later in time → negotiated over boilerplate); unresolved conflicts escalate to a human, never auto-resolve

Generate workflows — each obligation becomes a task DAG with owner, SLA, dependencies, and evidence requirements; emits BPMN 2.0

Execute — reminders at T−30d / T−14d / T−7d / T−1d, escalation ladder from owner → manager → legal → executive

Audit — append-only, hash-chained ledger enabling point-in-time reconstruction: "what did we believe was true on 2025-03-01, and why?"

Three guarantees: every obligation cites its source (provenance), conflicts resolve by explicit rules (precedence), and low-confidence or unresolved items route to a human (review).

How we built it

Phase 1 — Structure before intelligence. Before any LLM touched a document, we built the hierarchy layer. Layout-aware parsing (Docling for PDFs, python-docx for DOCX) preserved clause numbers, tables, and reading order. A Neo4j graph linked documents by parent–child, amendment lineage, and incorporation-by-reference edges. This was roughly half the correctness problem.

Phase 2 — Extraction with hard provenance. Our rule: no source span, no obligation. We used GPT-4o with structured outputs (instructor) so JSON parsing never failed, plus a verification pass that re-read the span and confirmed the extracted fields matched. Confidence was a composite: Phase 3 — Symbolic reasoning. Dates and precedence are never decided by a language model. We wrote a deterministic temporal engine over dateutil and workalendar, and a precedence engine over an ordered rule set. If no rule resolves a conflict, the system escalates — full stop. Phase 4 — Execution. Obligations became task DAGs, and DAGs became Temporal workflows. Compliance is inherently long-running: a 60-day notice is a 60-day timer that has to survive deploys and restarts. Deterministic task IDs meant recompilation after an amendment never duplicated work. Phase 5 — Audit ledger. Append-only, hash-chained Stack: Python, FastAPI, PostgreSQL + pgvector, Neo4j, Temporal, Redis, GPT-4o (structured outputs), LangGraph, React + TypeScript, Slack/Jira/email connectors, OpenTelemetry + Langfuse.

Challenges we ran into

Ambiguity in language. "Promptly", "reasonable efforts", "materially adverse" — no arithmetic exists for these. We couldn't extract deadlines and couldn't ignore them. The fix: model them as data-collection tasks. The system asks a human to interpret, records the answer as a Decision Record, and blocks dependent obligations until resolved. Turning ambiguity into a workflow instead of a guess was our biggest unlock.

Cross-document precedence. An Order Form says quarterly audits, the MSA says annual, the amendment is silent. Which wins? We built the ordered precedence engine and made rule six explicit: unresolved conflicts escalate to a human. Never auto-resolve. That single rule is why legal teams trusted the output.

Temporal arithmetic. "Within 30 days of the Effective Date" explodes into weekends, multi-jurisdiction holidays, timezones, and business-day vs. calendar-day ambiguity. We wrote a deadline DSL and a "show the math" UI — Effective 2025-01-15 + 30 days = 2025-02-14 — because users need to see why, not just what.

Alert fatigue. Our first reminder system generated 40+ notifications per contract per week. Users ignored all of them. We rewrote it with dedup, batching, priority tiers, quiet hours, and a tuned escalation ladder. The metric that mattered wasn't "notifications sent" — it was escalation precision (fraction of escalations actually warranted). Getting that above 80% was the difference between a tool people use and a tool people mute.

Evaluating correctness. There's no benchmark for "did the agent extract the right obligation?" We built one by hand: 200 contracts, two lawyers, inter-annotator agreement via Cohen's κ κ, extraction quality via

F

1

2 ⋅ precision ⋅ recall precision + recall F 1 ​ =2⋅ precision+recall precision⋅recall ​

with span accuracy tracked separately — a correct obligation with the wrong citation is still wrong for audit purposes.

Trust. The hardest challenge, and not a technical one. Legal teams don't adopt systems that make irreversible decisions. We had to design around human-in-the-loop by default: confidence gating, explicit abstention, mandatory review for conflicts, and a hard rule that no payment, termination, or waiver ever executes without human approval.

Accomplishments that we're proud of

End-to-end compiler working on real contracts — 4-document stack (MSA + Order Form + Amendment + DPA) compiled in ~6 minutes versus a 3–5 hour manual baseline

Zero hallucinated obligations in the auto-accept path, enforced structurally by the no-span-no-obligation rule

Deterministic temporal engine with exact-match accuracy ≥ 0.95 on our gold set — no LLM is allowed to compute a date

Explicit conflict resolution with full audit trail; every override produces a scoped, time-boxed, attributable Decision Record

Point-in-time reconstruction via hash-chained ledger — the one feature auditors actually ask for

A working review queue where human corrections become training signal and precedence decisions, so the compiler improves with every contract

Graceful abstention — the system says "I don't know" instead of guessing, and trust went up dramatically when we shipped that

What we learned

The hardest part of AI isn't the AI. Parsing, hierarchy, and provenance were harder than extraction. A better model on a bad document graph produces confidently wrong answers.

Abstention is a feature, not a failure. Early versions tried to resolve everything. Users didn't trust them. Once the system started saying "I'm not confident, please review this," adoption flipped.

Symbolic and neural reasoning are complements. LLMs are extraordinary at reading messy prose and terrible at arithmetic and logic. Rule engines are the opposite. The right architecture uses each for what it's actually good at.

Conflicts are data, not errors. Modeling conflict as a first-class entity with its own lifecycle — detection, precedence evaluation, resolution, decision record — is what made the system governable.

Provenance is a design constraint, not a feature. Requiring a span for every obligation forced grounding, eliminated hallucination, and made the output legally defensible.

Trust is earned through restraint. The most valuable AI systems don't replace human judgment — they make it scalable.

What's next for From Contract Language to Executable Business Actions

Near term (next 90 days)

Pilot with 200 contracts across two departments (Procurement + Security)

Expand doc-type coverage: NDAs, vendor agreements, employment contracts, regulatory filings

Ship the "show the math" UI for every computed date and precedence decision

Harden the review queue with keyboard-first workflows for legal reviewers

Medium term (6–12 months)

Cross-contract reasoning — detect obligations that reference other agreements ("as required under the Master Services Agreement")

Obligation graph analytics — surface risk concentration, single points of failure, and obligations that no team currently owns

Negotiation assist — flag clause language that will be hard to operationalize before signature, using the same obligation schema

Regulator mode — one-click evidence packets formatted for specific regulatory frameworks (SOC 2, HIPAA, GDPR DSARs)

Long term

Self-healing workflows — when a task consistently fails or escalates, the agent proposes a revised interpretation of the obligation and requests legal sign-off

Fine-tuned extractors per industry vertical, trained on the corrections from our own review queue

On-prem and air-gapped deployment for defense, healthcare, and regulated financial customers

Open obligation schema — publish the ontology so CLM vendors, ERP systems, and legal-tech tools can interoperate on a shared contract-as-code representation

The thesis stays the same: contracts are programs, compliance is the runtime, and the world needs the compiler.

Built With

• aws-security:-oauth-2.0-/-oidc
• aws-textract
• custom-precedence-engine
• cypher-ai-/-ml:-gpt-4o-(structured-outputs)
• deadline-dsl-data:-postgresql
• docusign-observability:-opentelemetry
• kafka-reasoning:-custom-predicate-engine
• languages:-python
• open-policy-agent
• prometheus-+-grafana
• pymupdf
• python-docx
• ragas-document-parsing:-docling
• react-flow-integrations:-slack
• salesforce
• saml-sso
• spacy
• sql
• tailwind-+-shadcn/ui
• tanstack-query
• typescript
• unstructured.io-backend:-fastapi