Project Journey: Malware Classification System

Inspiration

The rise of sophisticated malware threats motivated us to create an advanced machine learning-based malware classification system. Traditional methods struggle to detect modern threats, so we aimed to build a proactive solution that’s both powerful and user-friendly, accessible to both security professionals and non-technical users.

Key Learnings

Machine Learning: Enhanced our understanding of RandomForest, XGBoost, and handling imbalanced datasets using SMOTE.
Feature Engineering: Learned the impact of scaling, encoding, and polynomial features on model performance.
Model Stacking: Combined multiple models using ensemble techniques to improve accuracy.
User Experience: Built a user-friendly Dash interface to make the system accessible to non-experts.
Automated Reporting: Implemented automated PDF reports for easy access to results.

Development Process

Data Preprocessing: Handled missing values, scaled features, and used SMOTE for balancing the dataset.
Model Selection: Used RandomForest and XGBoost, stacking them to boost classification accuracy.
Evaluation: Assessed performance with confusion matrices and classification reports, and generated automated reports.
UI Development: Created a Dash-based interface for users to upload datasets, train models, and view results.
Real-Time Monitoring: Added progress tracking with visual feedback for an improved user experience.

Challenges

Data Imbalance: Solved using SMOTE to synthesize underrepresented samples.
Model Optimization: Hyperparameter tuning for XGBoost and RandomForest took substantial time.
Frontend-Backend Integration: Ensured seamless real-time updates between the backend ML pipeline and the frontend Dash interface.
Automated Reporting: Overcame difficulties in generating detailed reports using ReportLab.

Despite these challenges, we iterated on our design to build a robust malware classification system.

Built With

dash
git
joblib
matplotlib
python
randomforest
reportlab
scikit-learn
smote

Submitted to

MALWARE ANALYSIS HACKATHON @ IIT MADRAS

Created by

I handled the nitty-gritty of our project, like data preprocessing and cleaning, to make sure our data was in top shape. Once that was done, I jumped into coding the model and making it work smoothly. I also took the lead on our documentation because I believe in keeping everything clear and organized so we can replicate our work later.

Krishna Sharma
I contributed to the development of GuardNet AI, a machine learning-based malware detection system. My role included designing the system architecture, implementing advanced preprocessing techniques, and optimizing model performance with algorithms like RandomForest and XGBoost. I also integrated SMOTE to address data imbalance and built a user-friendly web interface using Dash for smooth user interaction. Additionally, I developed an automated reporting feature that provides detailed insights into model performance, ensuring the solution is both robust and accessible for end-users.

Prisha Sharma

Updates

Krishna Sharma started this project — Sep 21, 2024 02:14 PM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.