Spam Review Detection System

Inspiration

Online reviews play a huge role in consumer decisions, but the rise of fake and spam reviews has seriously undermined trust in platforms. Studies suggest that more than 20% of online reviews may be fake, especially in food and retail sectors. This inspired us to create a system that can automatically detect and filter spam reviews, protecting both consumers and businesses while preserving trust in online marketplaces.

What it does

The Spam Review Detection System identifies suspicious or fake reviews using a hybrid machine learning pipeline. It first detects anomalies in review behaviour and metadata, and then uses NLP models to analyse the actual text. The output is a classification of whether a review is likely spam or authentic.

How we built it

We built the system in two main stages:

1. Unsupervised Anomaly Detection: We applied models like Isolation Forest and Autoencoder on metadata (e.g., ratings, review length, timestamps, rating deviations) to flag unusual patterns. These generated pseudo-labels for spam vs. non-spam reviews.

2. Supervised Text Classification: Using the pseudo-labels, we trained NLP models (BERT and LSTM) on review text to capture linguistic cues of spam such as templated writing or mismatched sentiment.

The dataset was drawn from the Google Local Dataset (UCSD), and we engineered features across text, temporal, categorical, sentiment, and numeric dimensions to strengthen detection.

Challenges we ran into

Class Imbalance: Spam reviews formed only a small fraction (~5%), causing models like BERT to predict mostly the majority class.
Label Quality: Because we used pseudo-labels from anomaly detectors, some noise carried over into supervised training.
Computational Costs: Fine-tuning BERT was resource-heavy and limited experimentation.
Generalizability: The dataset focused on specific U.S. states, meaning the model may not fully capture cultural or linguistic spam variations globally.

Accomplishments that we're proud of

Successfully built an integrated pipeline combining anomaly detection with NLP classification.
Engineered a rich set of features (temporal, sentiment, rating deviations, exclamation use, etc.) that improved anomaly detection.
Achieved over 95.4% accuracy in the baseline BERT model (though with noted class imbalance issues).
Designed a fusion strategy to combine detectors, improving robustness and balancing precision vs. recall.

What we learned

Accuracy alone is not a good metric for highly imbalanced problems; precision, recall, and PR-AUC are much more meaningful.
Hybrid approaches (unsupervised + supervised) provide stronger results than relying on one method alone.
Feature engineering, even simple rule-based signals (like excessive exclamation marks), can significantly aid in spam detection.
Handling imbalance and noisy labels is one of the hardest but most important challenges in real-world ML.

What's next for Spam Review Detection System

Implement rebalancing strategies such as oversampling, class weights, or data augmentation for minority (spam) reviews.
Fine-tune and extend the NLP stage with models like RoBERTa or DistilBERT for better generalisation.
Explore semi-supervised and active learning to refine pseudo-labels with minimal human review.
Test and benchmark against more real-world datasets (Yelp, Amazon, Twitter) for broader applicability.
Develop monitoring dashboards for practical deployment to track model performance and adapt to evolving spam strategies.