ThreatStream

Real-time AI-powered cybersecurity platform combining Confluent Kafka streaming with Google Gemini AI to detect, analyze, and respond to threats in under 130ms.

Comment

Inspiration

Security Operations Centers are drowning in data. The average SOC processes over 10,000 alerts daily, yet 95% are false positives. Meanwhile, the average time to detect a breach is 207 days. We asked: what if AI could analyze threats in milliseconds instead of months? ThreatStream was born from this vision—combining real-time streaming with AI to give security teams superhuman detection capabilities.

What it does

ThreatStream is a real-time AI-powered cybersecurity threat detection platform. It ingests security events through Confluent Kafka, analyzes them instantly using Vertex AI, and surfaces actionable intelligence to analysts through a live SOC dashboard.

Key capabilities:

  • Streams security events through Confluent Cloud Kafka in real-time
  • Analyzes threats using Vertex AI (Gemini 2.0 Flash) in under 130ms
  • Maps attacks to the MITRE ATT&CK framework automatically
  • Visualizes global attack patterns on a 3D interactive globe
  • Provides AI-generated mitigation recommendations

How we built it

Data Pipeline: Confluent Cloud Kafka handles event streaming with three topics—raw logs, analyzed threats, and critical alerts.

AI Engine: Google Cloud Vertex AI with Gemini 2.0 Flash processes each event, providing contextual analysis, threat classification, and recommended actions.

Backend: FastAPI running on Cloud Run manages event processing, WebSocket connections for real-time updates, and API endpoints.

Frontend: React dashboard deployed on Cloud Run featuring real-time threat feeds, risk metrics, Kafka stream visualization, and a 3D attack surface globe.

Challenges we ran into

  • Vertex AI region configuration: Initially faced "unsupported region" errors until we properly configured the GCP_REGION parameter
  • Model versioning: Discovered Gemini 1.5 Flash was retired and had to migrate to Gemini 2.0 Flash
  • WebSocket persistence: Ensuring real-time connections remained stable across Cloud Run's serverless architecture
  • Cross-region latency: Optimizing Kafka message delivery between Confluent Cloud and GCP regions

Accomplishments that we're proud of

  • Achieved sub-130ms threat detection and analysis time
  • Built a production-ready full-stack security platform in under 2 weeks
  • Successfully integrated Confluent Kafka with Vertex AI for real-time AI analysis
  • Created an intuitive SOC dashboard that security analysts would actually want to use
  • Implemented MITRE ATT&CK framework mapping for industry-standard threat classification

What we learned

  • Real-time AI analysis is not just possible—it's transformative for security operations
  • Confluent Kafka's reliability is crucial for mission-critical security applications
  • Gemini 2.0 Flash provides remarkable speed without sacrificing analysis quality
  • Cloud Run's serverless model works excellently for event-driven security workloads

What's next for ThreatStream

  • Kafka consumer integration: Enable direct consumption from customer Kafka topics
  • Multi-tenant support: Allow multiple organizations to use the platform
  • Historical analysis: Add threat hunting capabilities over stored events
  • Automated response: Integrate with SOAR platforms for automated remediation
  • Custom AI training: Fine-tune Gemini models on organization-specific threat data

Built With

Share this project:

Updates