ThreatLens

Inspiration

Security teams deal with a large amount of activity every day. A single system can generate thousands of events, and finding the few events that actually deserve attention can take a lot of time.

Our S5 mini-project started from this problem. We wanted to explore whether AI could assist a security analyst instead of making them manually go through every event.

For ForgeHacks, we took that idea further and focused on making ThreatLens more useful as an AI-assisted security investigation tool. The goal was simple: take security events that are difficult to interpret on their own and turn them into information that an analyst can understand and act on.

What it does

ThreatLens is an AI-assisted security monitoring and investigation platform.

It collects and processes security events, identifies potentially suspicious patterns, and helps analysts investigate events from a single interface.

The workflow is:

Security Events → Processing → Threat Detection → Risk Analysis → Investigation

Instead of making an analyst manually search through large volumes of raw logs, ThreatLens highlights events that may require attention and provides context around them.

The analyst can investigate an event by asking questions such as:

What happened? — Understand the details and sequence of the event.

Is this activity suspicious? — Determine whether the observed behaviour shows indicators of a potential threat.

Why is it suspicious? — Understand the patterns or indicators behind the detection.

How serious is it? — Assess the potential risk associated with the event.

What should I investigate next? — Get relevant context for continuing the investigation.

What could be done in response? — Identify possible next steps for handling the incident.

ThreatLens uses XGBoost as part of its machine-learning-based threat detection pipeline. The resulting analysis is then presented through the application so that the analyst can understand the alert instead of looking only at a model prediction or raw log.

The AI component is used as an assistance layer within the security workflow rather than simply adding a chatbot to the application. The aim is to connect detection, context, and investigation in one place

How we built it

We built ThreatLens as a combination of security-data processing, machine learning, backend services, and a web-based interface.

The security events are first processed and converted into structured information that can be used for analysis. We then perform feature engineering to extract useful signals from the available security data.

For threat detection, we use XGBoost, which was trained to identify patterns associated with suspicious activity. The model produces a prediction that becomes one part of the overall investigation workflow rather than being treated as the final answer.

The application then connects the detection layer with the backend and frontend, allowing the analyst to view security events, examine the detected threat, and investigate the event through the ThreatLens interface.

The overall architecture is:

Security Events ↓ Data Processing ↓ Feature Engineering ↓ XGBoost Detection ↓ Threat / Risk Analysis ↓ Backend ↓ ThreatLens Interface ↓ Security Analyst

We kept the components separated so that the detection model can be improved or replaced without having to rebuild the entire application.

Challenges we ran into

One of the biggest challenges was dealing with security data that is not always clean or consistent. Security events can contain different types of information, missing values, and a large amount of normal activity alongside genuinely suspicious events.

Another challenge was false positives. In cybersecurity, detecting more events is not necessarily better. If a system generates too many unnecessary alerts, it can create another problem for the analyst.

We also had to solve the gap between an ML prediction and an actual security investigation. A model can say that an event is suspicious, but an analyst still needs to understand why it was flagged and what information should be examined next.

Integrating the ML component into the application and turning its output into something useful for a human analyst was therefore one of the more challenging parts of the project

Accomplishments that we're proud of

We are proud that ThreatLens goes beyond building an isolated machine-learning model and connects the model to an actual security investigation workflow.

Some of the things we are particularly proud of are:

Building an end-to-end security analysis workflow.

Integrating XGBoost into the application rather than keeping the model as a standalone experiment.

Processing security events and extracting useful features for detection.

Providing an interface where an analyst can investigate a detected event.

Focusing on reducing unnecessary investigation effort rather than simply generating more alerts.

Turning our S5 mini-project into a platform that we can continue developing beyond the hackathon.

Most importantly, we are proud of the idea behind ThreatLens: using AI to support the person investigating a security incident, rather than trying to replace them

What we learned

This project taught us that building an AI system is much more than training a model.

We gained practical experience with:

Security event processing

Data preprocessing and feature engineering

XGBoost-based classification

Handling imbalanced data

Connecting ML models with backend services

Building a security-focused user interface

Designing an investigation workflow

Thinking about false positives and analyst workload

Integrating different components into one working system

We also learned that the most useful AI systems are the ones that fit naturally into an existing workflow. A prediction by itself has limited value; giving the analyst enough context to understand and act on that prediction is what makes it useful

What's next for ThreatLens

ThreatLens is still a starting point, and there is a lot we want to build on top of it.

Our next steps include supporting more real-time security data sources, improving anomaly detection, and integrating external threat-intelligence feeds.

We also want to make the investigation experience more natural by allowing analysts to ask questions about an incident in plain language and receive answers based on the available security evidence.

Other areas we want to explore include:

MITRE ATT&CK mapping for detected activity

Automated incident summarization

Threat-intelligence correlation

More advanced anomaly detection

Response recommendations

Real-time event processing

Continuous model improvement

Our long-term goal is to evolve ThreatLens from a threat-detection system into a practical security copilot that helps analysts move faster from:

Detection → Understanding → Investigation → Response

Built With

• anomalydetection
• cybersecurity
• incidentinvestigation
• loganalysis
• naturallanguageprocessing
• python
• riskscoring
• securityanalytics
• securitymonitoring
• siem
• threatdetection
• threathunting
• webapplication
• xgboost