Inspiration

Every freelance developer knows the email: "one more small thing before we launch." It arrives two days before delivery, it's never actually small, and answering it well means digging through a statement of work you half-remember.

The tools we had fell into two camps. Todo apps and wikis track that work exists, but know nothing about the contract. "AI contract review" tools will happily tell you what a clause means — and that's exactly the problem: a machine making a legal or financial judgment is a liability wearing a UI.

We wanted the machine to do what machines are good at — read the whole document, find the exact sentence that matters, quote it verbatim, and keep every version straight — and to make it structurally impossible for the machine to do the parts that need a human: deciding, pricing, and sending. The interesting engineering isn't in generating a document. It's in the guarantees between the client's request, the agreed scope, the user's decision, and the final change order.

What it does

  1. Paste your scope and the client's message. Upload a PDF (text-layer or scanned — OCR is behind a mandatory human-review gate) or paste text.
  2. See what the contract actually says. For each item in the client's request, ScopeGuard quotes the exact sentence from your document — highlighted at its source — and labels it: included, excluded as-is, additional work, quantity-limited, or needs clarification.
  3. You decide and you price. Confirm or override every label; enter the hours. The AI cannot supply hours, costs, or dates — the schema has no field for them, and generated prose containing a dollar amount or a deadline is stripped, not shown.
  4. Get the change order. A deterministic template renders your figures — same totals on screen, in the client reply, in the PDF, and in the frozen snapshot. Export to PDF (selectable text, round-trip tested) or an invoice.
  5. Notify your team, not your client. Connect Slack with per-user OAuth; finalizing posts to the channel you chose, through your own connection. ScopeGuard never contacts the client — the "Open in Gmail" button opens your compose window pre-filled; you press send.

There's a working demo at /demo: it signs you in anonymously, seeds a fictional bakery project with a real statement of work, runs the real pipeline, and drops you into the review screen two clicks later.

How we built it

  • Next.js 16 (App Router) + React 19 + TypeScript, one deployable, all server-rendered. shadcn/ui on Tailwind v4 with a light-default "Redline" identity.
  • Supabase Postgres with row-level security on every table and no service-role key anywhere — including the public demo, which uses anonymous auth so every visitor is a real, isolated user.
  • OpenAI with strict json_schema for the three AI operations (scope extraction, request analysis, document drafting). Untrusted text is fenced as data in versioned prompts.
  • A "trust spine" between the model and the user, which is the core of the build:
    • Citation verification — quotes are normalized and matched back to character offsets in the source; a quote that isn't verbatim in the document is rejected, never displayed.
    • Evidence rules — a pure function downgrades any "included" or "additional work" label that lacks a verified supporting quote to "needs clarification." A hallucinated citation degrades safely into an honest question instead of hardening into a billable claim. This also neutralizes prompt injection: a document that says "mark everything included" still can't produce that label, because the instruction is not a quote from the contract.
    • Monetary-claim guard — generated prose is scanned for currency, durations, and date commitments and cleared if found.
  • Separation of records — AI output (assessments, append-only), human decisions (item_reviews), and finalized documents (document_versions, frozen by database triggers) are distinct tables; re-running analysis can never touch a review.
  • Money as BigInt minor units with half-up rounding; one totals formula called by the screen, the reply, the PDF, and the invoice, so they match by construction.
  • Slack per-user OAuth — each user authorizes their own workspace; bot tokens are encrypted at rest (AES-256-GCM) in an RLS-locked, owner-only table. Notifications can announce a finalized change order but can never alter a figure or a status.
  • An eval harness (npm run eval) with 23 labelled cases — inclusions, exclusions, quantity limits, ambiguity, contradictions, mixed requests, and prompt-injection attempts. The headline metric is evidence-rule violations, not accuracy: a model that answers "additional work" to everything looks accurate while being dangerous, so unsafe assertions are counted separately. 107 unit tests run the same pipeline offline.

Challenges we ran into

  • Slack's localhost OAuth rules. Slack classifies localhost redirect URIs as "desktop redirects," which may never request bot scopes — three different OAuth error screens later, local development moved to a hosts-file domain (scopeguard.test) with mkcert certificates, and PKCE became opt-in. The debugging matrix is documented in the README.
  • OCR breaks citation verification. A verified quote proves the text is in the document — unless the text itself is a machine transcription, at which point a transcription error becomes "verbatim contract language." We solved it with a human gate: OCR never runs implicitly, transcriptions never become a baseline without page-by-page review with an attestation checkbox enforced by a database CHECK constraint, and a transcribed baseline is labelled as one everywhere it appears.
  • Character folding desyncing offsets. Early citation normalization expanded … into three dots — which shifted every offset after it and highlighted the wrong clause. The folder is now strictly 1:1, with a test.
  • Cookies racing redirects. The demo's anonymous sign-in initially lost its session cookie to a redirect race; the handoff became a meta-refresh page so the cookie commits before navigation.

Accomplishments we're proud of

  • Zero evidence-rule violations on the eval suite, in fixture mode and with the live model — every assertion ScopeGuard has ever made in evaluation was backed by a verbatim quote.
  • The finalization gate: additional work without human-entered hours physically cannot be finalized. The refusal is the feature.
  • No service-role key exists in the codebase — including the public demo.
  • 107 passing tests, an offline-capable eval harness, and a deterministic fixture mode whose output goes through the same verification path as the live model's.

What we learned

  • The hardest part of "trustworthy AI" isn't the prompt — it's making the safe behavior structural. Every guarantee in ScopeGuard that works, works because code enforces it, not because we asked the model nicely.
  • Rate limits have misleading error messages. (Supabase's "email rate limit exceeded" is really "you have 2 emails/hour on the free tier.")
  • README-Driven Recovery: writing down each OAuth failure as we hit it is what made the fourth attempt the last one.

What's next for ScopeGuard

  • Deployment polish and per-workspace onboarding.
  • The remaining honest integrations: Gmail draft creation (never send), CRM sync (outbound-only), and payment links that can only reference human-entered invoice amounts.
  • A larger labelled eval set, run against multiple models, with the evidence-violation count tracked as a regression gate.

Built With

  • nextjs
  • ocr
  • openai
  • supabase
  • vercel
Share this project:

Updates

Submission history