NullPoint Agent

NullPoint proves exploitability instead of counting alerts: Nemotron extracts preconditions, a deterministic solver proves AWS network reachability. 11 Critical alerts collapse to 1 real ticket.

Comment

Inspiration

What it does

How I built it

Challenges I ran into

Accomplishments that I am proud of

Inspiration

Every security analyst knows the feeling: Monday morning, 400 new "Critical" findings, and a quiet certainty that 399 of them don't matter, with no proof of which one does. Alert fatigue isn't a tooling gap. It's a math problem: more alerts don't lower risk, they obscure it.

I wanted a tool with the courage to say no and the receipts to prove it.

What it does

NullPoint Agent is a vulnerability triage agent that falsifies exploitability instead of counting alerts. Feed it scanner findings plus AWS network telemetry, and it answers one question per CVE: can an attacker actually reach this?

The pipeline is deliberately adversarial: model proposes, code proves:

  1. NVIDIA Nemotron (via Nebius Token Factory) reads the CVE advisory and extracts structured exploit preconditions: port, protocol, authentication requirements, network vector, payload delivery.
  2. A deterministic graph solver (Python + NetworkX) evaluates security groups, NACLs, route tables, and load balancers to prove or disprove a network path from the internet to the vulnerable service.
  3. A fail-closed narrative guard cross-checks the model's exposure claim against the solver. The solver always wins: a blocked claim is logged as a caught model error. It is never acted on, and it can never veto a proof.

Then the product decides. Reachable findings go to a ranked action queue. Unreachable ones are dismissed into a tamper-evident, hash-chained dismissal log. And anything the math can't settle unknown hosts, data-plane payloads, model disagreements fails closed to a human review queue. Tavily enriches actionable tickets with live threat intel (CISA KEV status, observed weaponization) enrichment only; it never touches a verdict.

In the demo: 11 raw Critical alerts collapse to 1 real ticket.

How I built it

  • Nemotron 3 Nano (30B) through the Nebius Token Factory (OpenAI-compatible API) for precondition extraction with recorded fixtures so the whole pipeline runs in offline judge mode with zero credentials and zero network.
  • A hand-rolled AWS network model: security groups, NACLs, route tables, subnets, internet gateways, and ALB listeners evaluated hop-by-hop with per-hop allow/deny traces.
  • A read-only AWS importer (describe-* only — nothing in your account is ever modified) plus an Inspector2 findings converter, so the same pipeline runs against real infrastructure and real scanner output.
  • A dual-surface HTML report: the investigation trace on one side, the action queue vs. dismissal proofs on the other.
  • 52 tests, including regression tests for every fail-open hole I found and closed.

Challenges I ran into

The central fight was philosophical: the model is a brilliant extractor and a terrible decider. Early versions let narrative claims leak into verdicts. The fix became the product's spine, the guard, the solver-always-wins rule, and a review queue for everything uncertain.

External review then found three places where the product failed open: unknown hosts dismissed as "unknown host," the solver testing the model's guessed port instead of the scanner's observed one, and keyword-based payload classification. Each became a killed hypothesis with a regression test: unknown hosts, missing ports, and model/observation disagreements now fail closed to human review.

The last challenge was honesty in the demo fixtures: the recorded Tavily responses initially contained EPSS scores the live client can't return. I stripped them. If the offline stub shows something the live system can't produce, it's a lie and judges can tell.

Accomplishments that I am proud of

  • The refusal moment. F-001 is a pre-authentication RCE the model called "exposed." NullPoint dismissed it with proof: internal ALB, no internet ingress edge. No ticket. No page. No fire drill.
  • 9 of 11 model claims blocked by the guard in the demo run. The product catches the model being wrong on camera.
  • A real-account path, not just fixtures: read-only infra import plus Inspector2 findings ingestion, both fail-closed by design.

What I learned

  • Give LLMs the extraction job and never the decision job. They'll surprise you at the first and embarrass you at the second.
  • Fail-closed design is something evaluators feel: every uncertainty becomes a review item, never a silent dismissal.
  • The dismissal is the product. Anyone can file tickets; the courage to say "this is not exploitable, and here's the proof" is the scarce thing.

What's next for NullPoint Agent

Live Inspector2 and Security Hub wiring, multi-region import, action-queue delivery to Slack/Jira, and richer payload-delivery reasoning for data-plane paths. The thesis stays the same: fewer, truer alarms.

Built With

Share this project:

Updates

Submission history