SmartLogClassifier

• 

  “SmartLogClassifier UI — Upload logs and get instant intelligent labels.”

Inspiration:- Modern systems generate massive volumes of logs — some clean, many messy, and most unclassified. Manual log analysis is time-consuming, error-prone, and not scalable. We were inspired to create a hybrid, intelligent system that could automatically classify logs, combining the precision of regex with the deep understanding of transformers and LLMs.

What It Does:- SmartLogClassifier classifies system and application logs into meaningful categories using a three-tier classification engine:

Regex Rules – for high-confidence pattern-based logs

BERT Model – for semi-structured messages with context

LLM (Large Language Model) – for ambiguous or complex logs

It processes logs from various sources (CRM, HRMS, Billing, etc.) and tags them with labels like "User Action", "System Notification", "Workflow Error", or "Deprecation Warning".

How We Built It:- Python for core logic, Regex rules for common patterns, Sentence-BERT for contextual embeddings and classification, LLMs (via Groq API) for deep understanding of complex logs ,Streamlit for the user interface ,Pandas for data manipulation and file I/O, Hugging Face & Joblib for model loading and caching

Challenges We Ran Into:- Designing regex patterns that generalized without false positives,Managing slow LLM inference and fallback logic, Structuring the pipeline to balance speed and accuracy, Integrating different types models (rule-based, ML, LLM), Handling edge cases where even LLMs struggled to classify

Accomplishments That We're Proud Of :- Built a layered classification engine that gracefully degrades Created a working Streamlit UI for testing and demo Achieved high accuracy on structured logs using simple regex Successfully used LLMs for legacy/ambiguous log formats Separated classified vs. unclassified logs in outputs for transparency

What We Learned:- Regex is still powerful — if used wisely BERT is fast but needs clean, fine-tuned data LLMs can add serious value when traditional models fail Combining multiple approaches increases reliability Building modular code helps iterate quickly during a hackathon

What’s Next for SmartLogClassifier:- Add log anomaly detection and trend analysis Support live log streams (e.g., Kafka, Logstash) Build an admin dashboard for rule tuning and model feedback Extend classification to multi-label or hierarchical categories Deploy to the cloud with autoscaling (Streamlit Cloud or Hugging Face Spaces) Fine-tune the BERT model with real-world log data for higher accurac

Built With

• csv
• dotenv
• github
• groq-api-(llm)
• hugging-face-transformers
• joblib
• pandas
• python
• regex
• scikit-learn
• sentencetransformers-(bert)
• streamlit