Security Anomaly ML

Tagline: From network-flow predictions to deterministic analyst-facing security incidents.

Inspiration

A lot of machine-learning security work ends at the classifier: train a model, report precision and recall, and produce a score for each network flow.

But that is not where an analyst's work ends.

Even a good detector can still leave an analyst with thousands of individual alerts to investigate. I wanted to explore what happens when the ML model is treated as only one part of the detection pipeline rather than the final product.

What it does

Security Anomaly ML is an open-source network-flow detection system built around CICFlowMeter-compatible traffic.

It processes network flows through a causal temporal feature pipeline, scores them with a frozen Random Forest model, aggregates related detections into incidents, and promotes the incidents that meet a second decision threshold.

Instead of ending with a list of model predictions, the public interface produces deterministic incident-v1 JSONL objects intended to be easier for downstream security tooling and analysts to consume.

The current pipeline is:

CICFlowMeter flows → input validation → 128 causal features → ML scoring → flow alerts → incident aggregation → incident promotion → incident-v1

How I built it

The detector uses 128 model features:

76 CICFlowMeter flow features
9 static port/protocol behavioral features
43 causal temporal-context features

The temporal context is built only from traffic that occurred before the current flow. Same-timestamp flows are processed as peers before the state is updated, which avoids look-ahead leakage.

The classifier is a frozen Random Forest rather than a more complex deep-learning model. For this project, the larger engineering questions were temporal leakage, reproducibility, domain generalization, and the operational workload produced after classification.

The release includes:

Python CLI
Docker image
frozen and checksum-verified model
deterministic incident output
versioned feature/model/output contracts
reproducible public CI
offline inference with Docker networking disabled
archived software release with a Zenodo DOI

Validation

The evaluation was chronological:

January 22 → training
February 17 → validation
February 18 → locked temporal holdout

The February 18 holdout was opened only after the model, features, thresholds, aggregation policy, and promotion policy had been frozen.

On that locked holdout:

79,710 flow alerts
13,795 aggregated incidents
12,911 promoted incidents
98.37% flow recall
99.93% promoted incident recall
93.46% promoted incident precision
83.80% reduction from flow alerts to incidents
96.74% reduction in false-positive objects

Challenges

The hardest part was not getting the classifier to produce useful predictions.

The harder problems were preventing temporal leakage, keeping the evaluation honest, making the pipeline deterministic, and reducing thousands of individual flow detections into objects that are closer to what an analyst would actually investigate.

There is also an important limitation: the locked holdout comes from a later capture day, but it still belongs to the same dataset and network family.

That means these results should not be interpreted as proof that the detector will generalize to arbitrary real-world networks.

What I learned

The project reinforced something I increasingly see in security ML: improving another percentage point of classifier performance is not necessarily the most important next step.

A simpler model with controlled evaluation and a usable operational layer can be more informative than a much more complex model evaluated only as a benchmark.

The next important question for this project is domain shift — what happens when the detector sees traffic from an environment it has never seen before.

What's next

The current v0.1.0 release is intentionally research/evaluation grade rather than production-ready.

The next useful work is:

testing against independent network environments and datasets
measuring schema and exporter compatibility
reducing remaining analyst workload
evaluating incident aggregation under real traffic patterns
later adding service/streaming interfaces without changing the shared inference core

External testing and failure cases are more valuable at this stage than further tuning against the existing dataset.