Inspiration
Mainframes still run the core of banking, telecom, insurance, and retail. But real-time visibility into what’s happening inside them is extremely limited. Operators rely on batch SMF/JES logs, manual analysis, and decades of domain expertise to understand failures like ABENDs.
We wanted to show how modern cloud technologies — Confluent Cloud and Google Gemini — can bring mainframe operations into the real-time, AI-driven era. The idea: Stream mainframe-like events live through Confluent Analyze errors and ABENDs instantly with Gemini Visualize everything in a clean real-time dashboard
This bridges the gap between legacy infrastructure and modern AI.
What We Built
MainframePulse RT is a real-time mainframe intelligence system that turns operational events into instant insights.
Key features:
Live event streaming using Confluent Cloud (Kafka + SASL/SSL)
AI-powered diagnostics using Gemini 1.5 Flash
ABEND analysis
Root cause explanation
Recommended fix
Severity scoring
Confidence levels
Category classification
Real-time dashboard showing
Event feed
Severity breakdown
ABEND frequencies
Auto-scrolling logs
Click-to-expand detailed analysis panel
Start/Pause stream controls
Live indicator
Synthetic workload generator
Generates realistic SMF30, JES2, CICS, DB2, I/O, CPU spike events
15+ ABEND codes
Safely throttled to 5 events per 30 seconds
Everything runs live end-to-end.
How We Built It
Backend
Node.js + Express
KafkaJS consumer and producer
Confluent Cloud with SASL_SSL authentication
Gemini 2.5 Flash via the official Google Generative AI SDK
Custom event processor with in-memory ring buffer
Frontend (Dashboard)
Vanilla HTML/CSS/JS (no frameworks)
Auto-poll every 2 seconds
Live charts using
Animated table updates
Right-side event detail drawer
Real-time log panel
Data Pipeline Synthetic Events → Kafka Producer → Confluent Cloud → Kafka Consumer → Gemini Analysis → Event Processor → Real-Time Dashboard
What We Learned
How to set up secure Kafka streaming with Confluent Cloud (SASL_SSL)
Handling TLS issues and broker rotation with KafkaJS
Managing rate-controlled synthetic event streams
Using Google Gemini effectively for structured technical reasoning
Building a real-time dashboard without any frontend frameworks
Designing a full data pipeline from ingestion → AI → visualization
We also learned how powerful AI becomes when combined with streaming data — mainframe ops finally feel modern.
Challenges We Faced
TLS handshake failures when connecting KafkaJS to Confluent
Model migration issues — older Gemini models were deprecated
Producer/Consumer config mismatch (different config keys)
Stream rate control to avoid overloading Confluent or Google APIs
Designing a clean real-time UI with vanilla JS
Each issue taught us something new about building reliable streaming systems.
What’s Next
Integrate real SMF/JES logs
Add anomaly detection + predictive models
Add alerting & escalation workflows
Build multi-topic support for DB2, IMS, CICS, MQ streams
Package as a full SaaS for enterprise mainframe operators
Log in or sign up for Devpost to join the conversation.