AI Threat Intelligence Detection Model

Bar chart showing the distribution of benign and attack samples in the dataset
Top 10 features most influential in classifying network traffic as attack or benign.
Box plot showing the distribution of total forward packets for benign (0) and malicious (1) classes.
Visualization of total backward packets indicating variation between benign and attack traffic.
Comparison of maximum packet length between benign and malicious flows.
Heatmap showing correlation between top features and the label (Benign vs Attack)

Inspiration In today's interconnected world, cyber threats are becoming increasingly complex and frequent. Manual monitoring and rule-based systems are no longer sufficient to detect modern attacks. This inspired us to develop an AI-powered model that can intelligently classify network traffic as either benign or malicious using machine learning techniques. Our goal was to contribute towards building safer networks and improving real-time intrusion detection capabilities.

What We Learned Data Preprocessing: We learned the importance of handling missing values, irrelevant columns, and scaling features to improve model performance.

Model Training: We understood how Random Forest classifiers work for multi-class classification problems and how hyperparameters impact model accuracy.

Exploratory Data Analysis (EDA): Visualizing network data helped us discover underlying patterns, imbalances, and anomalies in the dataset.

Model Evaluation: We explored various evaluation metrics such as accuracy, precision, recall, and F1-score to assess the performance of the classifier.

Model Serialization: We learned to save trained models and scalers using joblib for easy reuse in production or deployment scenarios.

How We Built It Data Collection: Used the publicly available IDS Intrusion CSV Dataset from Kaggle (02-14-2018.csv).

Data Cleaning: Removed unnecessary columns, handled missing data, and applied label encoding.

Exploratory Data Analysis (EDA): Utilized seaborn and matplotlib to plot distributions and correlations to understand traffic behavior.

Model Training: Implemented a Random Forest Classifier using scikit-learn to detect benign vs. attack traffic.

Model Evaluation: Measured performance using various classification metrics.

Model Saving: Saved the trained model (ai_threat_intelligence_model.pkl) and the scaler (scaler.pkl) for future inference.

Challenges We Faced Data Imbalance: The dataset had an unequal distribution of benign and attack records, which impacted model learning.

Feature Selection: Deciding which features to retain was challenging as some were irrelevant or noisy.

Computational Resources: Training the model on a large dataset required optimization to avoid excessive memory and processing time.

Understanding Network Features: Gaining knowledge about network traffic features such as Flow Duration, Tot Fwd Pkts, etc., was necessary to interpret the dataset correctly.