Inspiration

In regulated industries, compliance review is still manual. An analyst reads a document and checks it against internal policy, rule by rule. AI could take the first pass, but two problems stop most teams.

Where the document goes.

  • Insurance: under the NAIC Model Bulletin on insurers' use of AI (adopted December 2023 and since adopted by many U.S. states), an insurer stays responsible for AI systems it gets from vendors. It is expected to do diligence on those vendors and to be able to audit them.
  • Health: a hospital can't put patient data (PHI) in a cloud service without a HIPAA business associate agreement.
  • Legal: law firms owe their clients confidentiality, and many require zero data retention from any AI vendor.

Whether the answer can be defended. Colorado's SB21-169 requires insurers that use external data and predictive models to test them for unfair discrimination and to document how they're used. A model that guesses, or a verdict with no evidence behind it, doesn't survive that kind of review.

We had already solved this for one sector. Our anti-money-laundering product, Sentinel PLD, runs an extract → compare → verdict pattern fully self-hosted, in active pilots. Sentinel Core generalizes that pattern to any regulated sector.

What it does

Sentinel Core checks a document against your organization's reference policy, the vault (a YAML file of rules), rule by rule. Each rule gets exactly one verdict, with quotes copied from the document:

  • no_cumple: does not comply.
  • revisar: needs review. The evidence is missing or conflicting, or a reference couldn't be confirmed, so a person decides.
  • cumple: complies.

The report lists first what needs a person. It records the document's SHA-256, the vault, the model and the time, and it downloads as JSON.

When a rule depends on an external reference that can go stale, such as a regulatory threshold, the engine makes one scoped Tavily search before asserting a failure. The query is built from the vault's template, never from the document, and the screen shows exactly what left.

To change sectors, you swap the vault. The engine stays the same.

Example: life-insurance underwriting (synthetic file, illustrative Mexican vault)

Rule Verdict Evidence
Tobacco declaration is consistent with lab results no_cumple "Tobacco use: No" is contradicted by "Cotinine (urine): POSITIVE (412 ng/mL)".
Sum insured is proportionate to declared income revisar MXN 3,000,000 against a declared income of MXN 150,000, with no justification on file.
Enhanced financial review above the threshold revisar Above the vault's MXN 2,000,000 reference, with no financial questionnaire attached. The reference is illustrative and Tavily found no public source confirming it, so Sentinel did not assume it.

The other three rules pass. The same engine reviews a vendor contract and an inpatient file using a different vault, with no code changes. In the contract, the governing law is outside the approved list and the liability cap is below one year of fees. In the inpatient file, amoxicillin is prescribed despite a documented penicillin allergy.

How a rule is decided

The model extracts facts, and each fact must be backed by a quote. Code then evaluates the vault's expression; the model does no arithmetic. The income rule is:

$$ \text{sum insured} \le 15 \times \text{declared income} \quad \lor \quad \text{justification on file} $$

In the sample, the ratio is

$$ \frac{3{,}000{,}000}{150{,}000} = 20 > 15 $$

and there is no justification on file, so the check fails. The vault marks this rule to fail to revisar, so a person makes the call.

A policy layer in code then applies three constraints:

  • every quoted passage must exist in the document;
  • a failure on an external reference stands only if a public source confirmed it;
  • a rule that errors fails closed to revisar.

The policy can only downgrade a verdict, never upgrade one.

Where the model runs

Where the model runs is a deployment choice, not a code change. Sentinel talks to one OpenAI-compatible endpoint, set by configuration. The engine classifies that endpoint as dedicated, shared or unverified, and the review screen says which, in plain language.

Data being reviewed Deployment Status
Synthetic or non-sensitive documents Nebius Token Factory, a shared endpoint. Fastest start. This is the live demo. The screen says "Shared endpoint… synthetic documents only."
Patient files, privileged contracts, underwriting files with personal data Nemotron on vLLM, on a dedicated Nebius AI Cloud GPU, single-tenant. Evaluated on one H100: 24 of 24 expected verdicts. Not the live demo.

Same engine, same vaults, one configuration change.

How we built it

  • Model: NVIDIA Nemotron 3 Nano 30B-A3B, a mixture-of-experts model with 30B parameters and 3B active. Reasoning is set per request: off for fact extraction, on for judging rules and public sources.
  • Nebius Token Factory: the live demo calls Nemotron 3 Nano through the Token Factory API. We developed and tested against it with synthetic documents.
  • Nebius AI Cloud: we served nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 with vLLM 0.30.0 on a dedicated gpu-h100-sxm VM. The BF16 weights are about 60 GB, so it needs one 80 GB GPU. We pointed the same code at it and re-ran the evaluation suite: 24 of 24 verdicts as expected, with one evidence warning (a cited quote differed slightly from the expected one; the verdict matched).
  • Engine: Python, rule by rule. A rule is either a check, where the model extracts quoted facts and an expression decides, or a criterion, where the model judges and cites quotes.
  • Verification: when a check fails on a rule with an external reference, the engine, not the model, runs a Tavily search. There is at most one search per rule and five per review, optionally restricted to allowed domains (gob.mx for the insurance threshold). The model judges the public snippets without seeing the document. A guard refuses any query that shares an 8-word run with the document.
  • Delivery:
    • FastAPI, with a web UI, a CLI and an API.
    • Visitor and admin logins, and a cap on reviews.
    • An audit log that holds hashes, verdicts and queries, but never document text.
    • An admin console for users, configuration and versioned vaults.
    • Docker behind Caddy, with HTTPS.
    • 461 automated tests that run with no network.

Challenges we ran into

  • Not guessing. The hardest part was getting revisar when evidence was missing or ambiguous, instead of a confident verdict either way. Prompting wasn't enough, so we enforced it in code.
  • Scoping the external call. The web should be reached only when the vault can't settle a rule on its own, and never with document content. Making the search the engine's decision, not the model's, kept every outbound query predictable and auditable.
  • Being honest about where the document goes. In the UI, a dedicated GPU and a shared API looked identical. So the engine now classifies its own endpoint, says so on screen, and refuses to call an address "dedicated" unless it provably is.
  • "On-prem" is the wrong word. We rent a GPU instead of owning hardware. What matters is that the compute is single-tenant, and that the operator can see which deployment they're on.
  • Real documents. Moving from plain text to real PDFs turned extraction into a problem of its own. There is still no OCR for scanned files.

Accomplishments that we're proud of

  • It runs end to end on the real model: through Nebius Token Factory with live Tavily searches, and on a dedicated Nebius AI Cloud H100. Not a mock.
  • When evidence was missing, it said revisar instead of inventing a verdict. That is what makes the output auditable.
  • The same engine handles three sectors (life insurance, vendor contracts, inpatient files) with zero code changes, only new vaults.
  • The screen tells the operator, in plain language, whether the document is going to a dedicated or a shared endpoint.

What we learned

  • A 30B mixture-of-experts model with 3B active parameters, on a single GPU, is enough for rule-by-rule compliance reasoning, as long as facts, arithmetic and quote checks are handled in code.
  • Keeping the "should we look outside?" decision in code, not in the model, made every outbound query templated and easy to audit.
  • The pattern isn't sector-specific. The hard part is data isolation and auditability, not the domain knowledge, which lives entirely in a swappable YAML file.

What's next for Sentinel Core

  • Pilots with regulated teams in insurance, legal and health, each on the deployment its data requires.
  • More sector vaults, and a larger evaluation set with negative cases (24 verdicts is a strong smoke test, not a benchmark).
  • OCR for scanned documents.
  • Recording the vault version on every report, so a verdict always names the exact policy it was judged against.

Try it out

https://sntnl.cc/app/ u: judge p: /r6LOBVt1x/DSW9aUVbWiKda

https://sntnl.cc/admin u: admin p: UMv6qtLPe5iwVvoW4OO1dC9i

Built With

  • fastapi
  • gpu
  • llm
  • nebius-ai-cloud
  • nebius-token-factory
  • nvidia-nemotron
  • openai-api
  • python
  • self-hosted
  • tavily
  • tool-calling
  • vllm
Share this project:

Updates

Submission history