Nerlo

Nerlo is the autonomous security registry for the agentic era: scanning, and scoring the MCP servers, skills, extensions, and rules that power AI agents, with every decision's reasoning public.

Comment

Inspiration

The idea took shape during a two-hour drive home while talking through the problem with Gemini.

I've spent 23 years in technology, including eight years running my own consulting firm and nearly five years at Amazon Web Services as a Senior Technical Trainer and Global Delivery Readiness Tech Lead. Today I run Learn 2 Cloud Training Canada out of Vancouver Island, delivering cloud architecture, security, and GenAI curriculum to enterprise engineering teams, and I'm partway through an MSc in Cyber Security at Royal Holloway, University of London. You learn where a technology breaks by teaching it. The questions engineers ask when they're two weeks from shipping tell you exactly which part nobody has solved.

When the Model Context Protocol (MCP) and autonomous agent frameworks exploded, the pattern was immediately familiar: developers were pulling unvetted third-party tools into production environments with zero review. It was the npm supply-chain crisis all over again, but with a critical difference. These packages have autonomous agency. They execute code, call other tools, access filesystems, and reach the network on the agent's initiative rather than a human's.

While building NubiferOS, a security-hardened Linux distribution where I used LLMs to generate roughly 39,300 lines of code, I realized that AI writes software far faster than human security teams can review it. The bottleneck in AI development isn't code generation anymore. It's verification at machine speed.

Traditional package registries rely on human maintainers. The agentic era moves too fast for that. By the time I got home, the vision was clear. We needed a security registry that audits at the speed the ecosystem publishes, operated autonomously by AI, with deterministic math and a public audit trail.

What it does

Nerlo is the autonomous security registry for the agentic era: continuously discovering, scanning, and scoring the MCP servers, skills, extensions, and rules that power AI agents, with every decision's reasoning public.

It runs continuously and unattended. As of August 16, 2026, the registry tracks 37,946 packages across four distinct formats (MCP servers, Claude Skills, Gemini Extensions, and Cursor Rules), of which 28,061 have been scanned at least once. Nobody curated that corpus. An AI found it, classified it, and decided what entered the pipeline.

Nerlo publishes the shape of its own coverage rather than only its totals. Of the packages tracked but never scanned, 9,276 are queued behind a first scan and 609 cannot be acquired at all. That second number is a permanent hole in the coverage claim, and it is on the coverage page because a registry that reports only what it managed to reach is reporting a sample and calling it a census.

The part that matters most is what Nerlo refuses to do.

Every other security badge is an oracle: one opaque score, take it or leave it. Nerlo is an aggregator. Twelve scanners examine each package, and every scanner's scorecard is published side by side, at one flat rank. No scanner can veto another. No scanner gets visual priority. A composite score exists as a convenience, but it can never bury a single scanner's warning.

And for most of the registry, Nerlo publishes no composite verdict at all. Only 2,069 packages carry an attestation, meaning a namespace their publisher proved they own in the official MCP registry, or a first-party source read directly. The other 34,837 have nothing attesting to them, and for those the composite reads Unrated. Each scanner's findings are still published in full. The single aggregate verdict is withheld.

That is a deliberate stance about authority, and it is expensive. It means Nerlo declines to issue a summary judgment on 94% of the packages it has done the work to scan. A per-scanner finding is a checkable statement about what a tool reported. A single unified verdict is a judgment about someone's work. Nerlo publishes the first and withholds the second from any publisher who never engaged with it, because these are real repositories belonging to real developers, most of whom never asked to be rated.

The model. The public registry is free forever, for everyone, without an account. Scores, per-scanner findings, audit reports, and the AI decision log are all open. Maintainers never pay to understand their own score. Revenue comes from the layer above the data: API access, scan history, and governance for teams that need to enforce policy rather than just read it, priced per seat from $19 to $79 per month with an enterprise tier for regulated sectors. The free registry isn't a loss leader. It's the proof of authority the commercial product depends on, and it stays free because a security signal that costs money to read is a security signal nobody checks.

How we built it

AI operates the registry. That is the architecture, not a feature.

Autonomous discovery via Gemini

Running through Vertex AI, Gemini continuously scours repositories, package managers, and developer hubs, classifying every candidate it finds and deciding what enters the pipeline, with no human in the loop. In the trailing seven days, the platform recorded 68,798 autonomous AI decisions.

Autonomous operations with public accountability

Six modules make operational calls without a human: publish, archive, remove, quarantine, self-repair, and an emergency brake. Each one writes its reasoning to a public audit trail. The division of labor is intentional. A deterministic rule engine makes the decision so it is reproducible, and the AI explains it so it is accountable.

The registry's first autonomous decision was a rejection, and it is on the record:

"The server was rejected from the registry because its composite badge was 'Unsafe' with a composite score of 50.0. […] Had the composite badge been anything other than 'Unsafe', the server would have been published."

Scanning every distribution surface

A single MCP server rarely ships from one place. The same package can reach you as git source, a GitHub release asset, an npm tarball, or a pip wheel, and those copies do not have to be identical. Nerlo acquires every distribution surface it can find and scans each one independently, inside digest-pinned Docker-in-Docker containers with the target mounted read-only and the network interface disabled. Acquisition never executes package code: npm tarballs are pulled with --ignore-scripts, wheels are unpacked rather than built, and every surface is reduced to a tree of inert files before a scanner reads it.

Two phases run in production today:

  1. Static analysis. Permission mapping, malicious-pattern detection, and taint analysis for secret-to-network egress.
  2. Dependency auditing. Deep supply-chain vulnerability scanning.

Twelve scanners publish verdicts across those phases. Nine are off-the-shelf tools pinned and run as-is: AgentShield, agent-audit-kit, Bearer, Capslock, Cisco Skill Scanner, govulncheck, OSV-Scanner, Trivy, and Trivy run a second time against the container image a package ships. Three are Nerlo-original, labeled as such everywhere they appear, because you should never have to guess which verdicts came from a third party and which came from me.

Surface divergence, the signal I built for

Holding more than one surface for the same package makes a specific attack visible: behavior present in the published package but absent from the source it claims to be built from. An npm tarball or release asset containing network calls, credential reads, or obfuscated logic that never appears in the git tree.

That is exactly the shape of the postmark-mcp and FakeGit incidents, where the repository looked clean and the shipped artifact did not. A methodology that scans only the source repository is blind to that class of attack by construction. The divergence itself is published as a finding rather than averaged into a score.

Deterministic trust scoring

The composite is a deterministic weighted mean over five components, renormalized across only the phases that produced a report:

$$T = 0.25\,s_{static} + 0.25\,d_{depend} + 0.25\,f_{fuzz} + 0.15\,a_{deep} + 0.10\,m_{vitality} - \sum \Delta_{behavior}$$

In production, that renormalization matters more than the weights do. Only static analysis, dependency auditing, and maintenance vitality produce reports today, so those three carry the entire score. The fuzzing and deep-behavioral weights, and the behavioral deduction term, are reserved for phases that are not running. A phase that hasn't been built is dropped from the average entirely and never counted as a zero, so a clean partial scan isn't dragged down by work I haven't done.

Two floors then sit on top of the score, and both can only ever make a badge worse. The composite badge is never greener than the worst verdict any scanner that completed handed down. And a composite that would otherwise read Clean is capped at Caution once findings across all completed scanners accumulate, counting each high as two and each medium as one, to six or more. Averages dilute; without the floor, a majority of clean scanners could arithmetically outvote the one scanner that found something serious.

Every constant, threshold, deduction, and floor is published on the methodology page so anyone can recompute a score by hand and get the same number. Determinism is the point: the same findings always produce the same score.

Auditing the auditor

Nerlo publishes its own threat model, scanner provenance, and SBOM, including a verification checksum for every scanner in the image and the exact image digest scans execute in. The attestation endpoint refuses to serve if a digest pin is broken. The emergency brake's state is public. Every scan stores its per-scanner reports, component scores, and the exact weights applied, immutably for 24 months, because a score you cannot recompute is a score you have to take on faith.

Challenges we ran into

The hardest problem was deciding what not to build.

The original design called for four phases, including adversarial prompt-injection fuzzing and deep behavioral execution, meaning actually running packages and watching them misbehave. Both are technically achievable. I formally ruled against both, and that turned out to be the most important engineering decision in the project.

Fuzzing an MCP server properly means sending traffic to a live endpoint. Nerlo doesn't own those endpoints. A company whose product is a security badge, sending unsolicited probe traffic to third-party infrastructure, is a liability with no requirement authorizing it. The alternative was building a mock from the declared schema and probing that, and it fails on correctness: it scores the mock, not the package.

Deep behavioral execution ran into something more sobering. I measured containment and found Firejail failing open inside the scan container, exiting with code 0 while the supposedly sandboxed process still reached the network. I was one bad assumption away from shipping "isolated execution" that wasn't isolated. So I didn't ship it.

The lesson generalizes. A security product's failures are asymmetric. A missed finding disappoints; a false claim about containment is a breach sold as a feature. Two phases that provably work across 28,061 packages are worth more than four phases where two are aspirational. Nerlo documents which phases run, publicly, on the methodology page.

The second challenge was calibration honesty, and it ended in a rename. The badge ladder originally read Verified, Caution, Unsafe. Measuring my own output showed both ends overclaiming. "Unsafe" only ever meant that at least one scanner scored a package below 60, which on this corpus was true of packages scoring 95 out of 100 with no critical finding anywhere. "Verified" failed in the other direction, because Nerlo verifies nothing: it runs tools and publishes what they found. The ladder now reads Clean, Caution, Flagged, with the underlying API values unchanged so nobody's integration broke. Only the words a human reads changed, because those were the ones making a claim the evidence didn't support.

The third challenge was discovering that discovery was the weak link. In July I audited Nerlo's own crawler against 39 known first-party vendor MCP servers, and the results were bad in a useful way. Of the 32 that resolved to a live repository, only 13 carried the GitHub topics the crawler searched on, meaning 19 of 32 were unreachable at any page depth. The topic mcp-server alone held 21,625 repositories, and a single crawl keeps at most 500 of them ordered by most recently updated: 2.3% of one topic, biased toward whatever was pushed to last, so a stable finished package sinks out of reach precisely because it is stable and finished. Package-registry crawling covered npm, PyPI, and crates only, invisible to the 11 of 39 that ship exclusively as container images or release binaries. And no authoritative first-party index was being consumed at all.

Every one of those numbers is published on the coverage page, including the 15 first-party packages Nerlo still has no match for, with the matching rule deliberately set so the error runs against us. Auditing an ecosystem obligates you to audit your own instrument first, and the honest version of that is a page listing what you missed.

Accomplishments that we're proud of

  • A registry that runs itself. 37,946 packages discovered, classified, and queued, 28,061 of them scanned, backed by 68,798 autonomous AI decisions in seven days, every one logged with its reasoning, queryable, and exportable. No human curates what enters the registry or what it scores.
  • Refusing to be an oracle, at cost. Twelve scanners published flat, no vetoes, no hierarchy, and no composite verdict at all for the 34,837 packages nothing attests to. That is 94% of the registry where the work was done and the summary judgment was withheld on purpose.
  • Auditing my own auditor. A public threat model, per-scanner provenance with checksums, digest pins enforced at serve time, a published SBOM, an emergency brake whose state is visible, and a coverage page that names what the crawler missed and why.
  • Renaming badges after measuring that they overclaimed. Nerlo caught its own labels making claims the underlying evidence didn't support and fixed the words rather than defending them.
  • Choosing accuracy over an impressive spec sheet. I cut two of four planned phases after measuring that I couldn't do them safely or correctly, and documented why dynamic execution is held back until containment can be verifiably enforced.

What we learned

Teaching GenAI, data engineering, and DevOps to global enterprises taught me that organizations want to move fast and are terrified of data leaks and compliance failures. Building Nerlo sharpened that into something more specific. Traditional security tooling is blind to agentic risk. You can write mathematically flawless code that is still fully exploitable through prompt injection, over-permissioned tool grants, or install-time instructions aimed at the model rather than the machine. The attack surface moved from the code to the instructions, and no CVE database covers it.

The deeper lesson was epistemological. Building a system that rates other people's work forces you to be precise about what you actually know. A scanner finding is evidence. A trust score is an opinion. Publishing the second as though it were the first is how security registries become the thing developers route around. Separating the two produced a registry where every autonomous decision arrives with its reasoning attached, on a record anyone can read without an account.

There is a governance lesson in there too, learned the awkward way. Nerlo ran an appeal process for publishers who thought a badge misrepresented them, and I withdrew it. An aggregator that moves a score because the package's author asked is not worth reading; what makes the number trustworthy is that nobody, including me, can talk it into changing. It moves when the code moves. The overrides made under that process stay on the record with the operator's identity and written justification attached, because withdrawing a process does not un-say what it already said.

And the premise NubiferOS hinted at is now verified, in the most direct way available. Nerlo is one person and a fleet of coding agents. A registry indexing 37,946 packages, running twelve scanners, and publishing a full methodology was built by a solo operator working the way the ecosystem it audits works. That is the thesis demonstrated rather than argued: AI builds software faster than humans can verify it, which is why verification itself has to be autonomous.

What's next for Nerlo

  • Closing the discovery gap. Reading authoritative first-party indexes rather than searching self-described metadata, and adding the container-image and release-binary channels that make 11 of 39 known first-party servers invisible today. The coverage page tracks the 15 still unmatched, and the goal is to drive that to zero in public.
  • Static context-window analysis. Expanding static analysis over MCP tool descriptions and system instructions. A tool description is text loaded into a model's context and followed, making it artifact-derived evidence readable with no endpoint, no sandbox, and no probe traffic. That is where the highest-leverage coverage gains are, and it is the direction I'm actively building.
  • Enterprise private registries. Private repository scanning, approval workflows, and audit-ready compliance reporting for regulated sectors where agents cannot execute unreviewed code. I'm pursuing design partners now, and the enterprise intake is live.
  • Dynamic phases, if and only if containment is provable. The fuzzing and deep-behavioral weights stay reserved until sandbox escape can be measured rather than assumed. If that day doesn't come, the weights stay at zero and the methodology page keeps saying so.
  • Revenue posture. Nerlo is pre-revenue by design during the competition window. The registry launched free and public and stays free through judging, because judges and developers testing the product should not hit a paywall. Marketing and customer-acquisition spend to date is $0.

Built With

Share this project:

Updates