Mcp/cls/claude code features
InstallGate_APIs
InstallGate_Allow
InstallGate_misspelled_package
InstallGate_Json

Inspiration

AI coding agents now write code, choose dependencies and install packages with little human intervention. When an agent meets a package name in generated code or broken documentation, it tends to assume the package is legitimate simply because it exists, or tries to install a name that was never real.

That turns into a supply-chain risk. Hallucinated dependency names can lead agents to typosquatted or malicious packages, a single compromised release can slip into an otherwise trusted project, pickle-based model weights can run code when loaded, and outdated container base images keep known vulnerabilities alive. On October 21, 2026, PHP shipped emergency releases (8.2.35, 8.3.36, 8.4.27, 8.5.12) for two critical remote-code-execution flaws, CVE-2026-109618 and CVE-2026-109619: any Dockerfile pinned to an older tag was exposed.

The problem is not a lack of security scanners. It is the gap between detecting a risk and enforcing a decision at the exact moment an AI agent tries to install or load a dependency.

We built InstallGate to close that gap: a lightweight, auditable checkpoint that evaluates dependency trust before installation, explains every decision with verifiable evidence, and keeps a human in control of remediation.

What it does

InstallGate is a pre-install trust gate for AI coding agents.

It accepts pip install and npm install commands, Hugging Face model repositories and Dockerfile FROM instructions, and returns one of three verdicts:

ALLOW: the configured checks found no blocking evidence.
REVIEW: the evidence is inconclusive, or the target needs a human look.
BLOCK: deterministic evidence satisfies a blocking policy rule.

Every verdict carries numbered evidence references such as [EV-1] and [EV-2], so anyone can trace the reasoning.

InstallGate reads metadata only. It never installs packages, never unpickles model files and never executes the commands it suggests. When a dependency is blocked, it can propose a pinned, re-checked alternative as a command diff, which waits for approval by a human or the calling agent:

- pip install reqeusts
+ pip install requests==<verified-version>

The same GateVerdict contract powers three entry points:

HTTP API: POST /check
CLI: installgate check "pip install ..."
MCP tool: check_install, so coding agents can request a check mid-workflow

Input	Verdict	Reason
A package name that does not exist on PyPI	BLOCK	The registry confirms the package cannot be found
`reqeusts`, registered recently	BLOCK	Close lookalike of `requests` plus a brand-new project
`litellm==1.82.7`	BLOCK	Named in a version-specific OSV malicious-package advisory
`litellm==1.82.6`	ALLOW	Same package, a version the advisory does not cover
Hugging Face repo with pickle-only weights	REVIEW	Deserialization risk; Safetensors is preferred
`FROM php:8.3.30-apache`	BLOCK	Below the patched floor for the PHP 8.3 branch (8.3.36)
`FROM php:8.3-apache`	REVIEW	A floating tag does not identify a fixed image version
A pinned, established FastAPI release	ALLOW	No configured risk evidence; no model call needed

An ALLOW means the configured checks found no reason to block the target. It is not a guarantee that the dependency is safe.

How we built it

One principle shapes the design: code establishes the facts, policy sets the verdict, the model explains the evidence, and a validator enforces the limits. The LLM is never the authority on whether a dependency is safe.

1. Deterministic analysis engine. Typed parsers turn raw commands into structured targets. Read-only clients query PyPI, npm, OSV.dev and the Hugging Face Hub through an explicit host allowlist, with per-call timeouts and a SQLite cache. The engine collects content-hashed evidence:

package and release existence, project and release age, maintainer-change signals;
OSV MAL- records, matched against affected versions and ranges, not just names;
name similarity (Damerau–Levenshtein) against a pinned top-5,000 PyPI snapshot;
Hugging Face weight formats;
Docker base-image tags against branch-specific patched-version floors.

A YAML policy (rules P1–P10) maps evidence to a verdict. A BLOCK requires at least one hard, deterministic piece of evidence; uncertain signals can trigger review but never a block on their own.

2. Nemotron on Nebius Token Factory. We call NVIDIA Nemotron models through the OpenAI-compatible Token Factory API, each with a narrow job:

Nemotron-3 Nano extracts extra candidate targets from unstructured input, and may only add targets that appear verbatim in it;
Nemotron-3 Super writes the human-readable explanation, citing evidence ids;
Nemotron-3 Ultra is reserved for REVIEW cases with a risk score of at least 0.6.

Clean ALLOW decisions make no model call. Each verdict includes a cost receipt with a counterfactual "all-Ultra" estimate, so the cost of extra analysis stays visible. Tavily searches allowlisted security domains for recent advisories; those web results are stored as untrusted, supplementary evidence and can never trigger a BLOCK alone.

3. A validator that limits model authority. Any explanation sentence without a valid evidence citation is dropped. Verdicts are ordered

$$\text{ALLOW} < \text{REVIEW} < \text{BLOCK}$$

and the final decision is

$$\text{final} = \max(\text{policy verdict},\ \text{permitted model escalation})$$

The model may raise ALLOW to REVIEW, but it can never lower a deterministic verdict or issue a BLOCK. If a registry, Tavily or the model is unavailable, InstallGate records the limitation and fails closed to at least REVIEW; a BLOCK already backed by hard evidence stays in force.

4. Safe Mode remediation. Substitutes come from a closed candidate set, are pinned, and pass the same deterministic checks before they appear as a command diff. InstallGate never runs them.

5. Contracts, stack and workflow. Python 3.11, FastAPI, Typer, the MCP SDK, Pydantic v2 contracts mirrored in Zod, SQLite, and a Next.js UI that shows the same install with the gate off and on. Two engineers built in parallel against a data contract frozen on day one, with one task per branch and pull request, and CI (Ruff, mypy, pytest, Gitleaks, UI build) required for every merge.

6. Reproducible evaluation. 150 labelled cases, metadata only, run offline:

Class	Cases	Source
Benign	44	Pinned and unpinned PyPI installs, Safetensors Hugging Face repos, patched Docker tags
Malicious	50	Real OSV `MAL-` records from `ossf/malicious-packages` at a pinned commit
Hallucinated	30	Names verified to return 404 on PyPI at validation time
Typosquat	20	Near-misses of the top-5,000 PyPI names
HF pickle-only	5	Repos with pickle weights and no Safetensors file
Floating Docker tag	1	A base image with no fixed patch version

Metric	Target	Result
Block recall	≥ 95%	100%
False-block rate on benign	≤ 2%	0%
Review rate on benign	—	11.4%
Deterministic-path latency, p50 / p95	< 1.5 s / 4 s	0.02 / 0.66 ms

Latency covers the local deterministic decision only, not registry requests or LLM-assisted analysis. These results describe this evaluation set and its labels; they are not universal detection rates.

Challenges we ran into

Our first benchmark graded itself. The first test set was written from the same assumptions as the code, so a perfect score proved nothing. We rebuilt it from real OSV advisories at a pinned commit, a real top-5,000 snapshot and registry-verified names, and started recording which rule decided every case.

Removed malicious packages hid the OSV check. Malicious packages are usually deleted after they are reported, so a live check returns not found before the OSV rule ever runs. We split results by registry state and added clearly labelled pre-removal simulations so the OSV path is actually exercised.

Name-level matching blocked clean versions. An advisory covering litellm named only a few specific releases, but matching by name blocked every version. We now match exact versions against affected lists and ranges, treat {"introduced": "0"} as "all versions", and keep ranges we cannot order reliably (such as Git commit ranges) as supplementary evidence instead of guessing.

One version floor let vulnerable PHP through. A single minimum version per image compared releases across branches and let an unpatched tag pass. We moved to per-branch floors, verified against the official PHP security advisory.

"I couldn't check" had no shape. For npm (no vetted popularity baseline) or floating Docker tags, the gate returned REVIEW with nothing to cite. We added a check_unavailable evidence type and rule P10 through a reviewed contract change, and taught the contract to pin Hugging Face models by Git revision, without changing any existing evidence hash.

Labels went stale. Between runs, 5 of our 30 "hallucinated" names turned out to exist on PyPI, and two had been registered within the previous three months. That is the threat model happening in real time; we now re-verify names before every run and record each case's source and verification time.

Fresh releases are a real trade-off. Unpinned installs of FastAPI, NumPy, Django, Pydantic and SQLAlchemy go to REVIEW because their latest release is under 30 days old. We kept the threshold explicit: a fresh release of an established package is exactly how a compromised version arrives, and release age is a reason to look, not proof of malice.

Accomplishments that we're proud of

100% block recall and 0% false blocks on our 150-case evaluation set, under its defined labels and conditions.
Deterministic policy keeps authority over blocking; the LLM is limited to explanations and permitted escalation.
No automatic execution: no installs, no unpickling, no running proposed commands.
A local deterministic path that needs no LLM call for clean ALLOW decisions.
One verdict contract shared by HTTP, CLI, MCP and the UI, so the interfaces cannot disagree.
Evidence-linked explanations instead of unsupported model assertions.
A reproducible evaluation that records data provenance and re-verifies time-sensitive labels.
A UI that makes the trade-off visible by showing the same install with the gate off and on.

We are proudest of the boundary itself: the model helps explain the result, while the policy and validator define what it is allowed to do.

What we learned

Let code decide and let the model explain. LLMs make security decisions easier to understand, but enforcement has to be deterministic when a wrong call can introduce a supply-chain compromise.
A version is not a package. Advisories usually cover specific releases or ranges; matching by name blocks legitimate dependencies and erodes trust in the gate.
Benchmarks need provenance. Registries change and labels decay, so every case must record where it came from, when it was verified and which rule decided it.
"I don't know" is a feature. A gate that reports an unavailable check is more trustworthy than one that treats missing evidence as safety.
Security and usability trade off. Conservative thresholds reduce risk but add review friction; the policy should make that trade-off explicit.
Contracts are engineering tools. Freezing the data contract on day one let two engineers build separate stages without the terminal, API, MCP tool and UI drifting apart.

What's next for InstallGate

Broader coverage: more registries, model formats and container ecosystems, starting with a vetted npm popularity baseline.
Fresh intelligence: continuously refreshed advisory feeds and known-good version data, while keeping evaluations reproducible.
Agent-native enforcement: deeper integration into coding-agent workflows, with clear human approval paths for risky decisions.
Stronger evaluation: more independently sourced malicious packages, adversarial cases and time-based tests for newly registered packages.
Production readiness: policy customization, audit trails and deployment options for teams running many agents across many repositories.