DeceptiGuard - Devpost Submission
Inspiration
The inspiration for DeceptiGuard came from a personal experience of being misled by fake reviews while booking a hotel. After arriving at what was supposed to be a "5-star luxury resort" based on glowing reviews, we discovered it was far from the promised experience. This led us to research the scope of fake reviews - and we were shocked to learn that up to 30% of online reviews are deceptive, costing consumers billions annually and undermining trust in e-commerce platforms.
We realized that while humans struggle to identify sophisticated fake reviews, AI could detect subtle patterns in language and imagery that reveal deception. With the rise of multimodal AI and transformer models, we saw an opportunity to build something that could protect both consumers and legitimate businesses from the growing threat of review manipulation.
What it does
DeceptiGuard is an AI-powered web application that instantly classifies customer reviews as either truthful or deceptive with 85%+ accuracy. The system works in two modes:
Multimodal Analysis:
- Analyzes review text using Microsoft's DeBERTa-v3 transformer model
- Processes accompanying images through ResNet18 or color histogram analysis
- Fuses text and visual features for enhanced detection accuracy
Real-time Detection:
- Users can input review text and optionally upload images
- Receives instant classification with confidence probabilities
- Provides detailed analysis of why a review might be deceptive
Key Features:
- Live training interface with real-time progress tracking
- Support for custom datasets (Amazon reviews, hotel reviews, etc.)
- Model persistence - save and reload trained models
- Comprehensive evaluation metrics (precision, recall, F1-score)
How we built it
Technology Stack:
- Frontend: Streamlit for rapid prototyping and user-friendly interface
- AI Models: Microsoft's DeBERTa-v3-base for text analysis, ResNet18 for image features
- Backend: PyTorch for model training and inference
- Data Processing: Pandas, scikit-learn for data handling and evaluation
Architecture:
- Text Pipeline: Raw text → preprocessing → DeBERTa tokenization → 768D embeddings
- Image Pipeline: Images → ResNet18/ColorHistogram → feature extraction → 512D vectors
- Fusion Layer: Concatenate text + image features → classification head → probabilities
Development Process:
- Started with benchmark datasets (Amazon fake reviews, deceptive opinion corpus)
- Implemented text-only baseline using DeBERTa-v3
- Added multimodal capabilities with computer vision components
- Built interactive Streamlit interface for training and inference
- Optimized for real-time performance and scalability
Technical Innovations:
- Custom preprocessing pipeline for review text normalization
- Flexible fusion architecture supporting text-only or multimodal modes
- Real-time training visualization with live metric updates
Challenges we ran into
Dataset Complexity: The biggest challenge was handling diverse review formats and ensuring balanced datasets. Real-world fake reviews are sophisticated and constantly evolving, making it difficult to create robust training data that generalizes well.
Multimodal Fusion: Combining text and image features effectively required extensive experimentation. We had to balance the contribution of each modality and prevent one from dominating the classification decision.
Model Performance vs Speed: Achieving high accuracy while maintaining real-time inference speed was challenging. DeBERTa-v3 is computationally intensive, so we had to optimize the pipeline for practical deployment.
Memory Management: Training large transformer models with image data required careful memory management and batch optimization to prevent out-of-memory errors during training.
Evaluation Methodology: Ensuring fair evaluation across different review domains (hotels, products, restaurants) while avoiding overfitting to specific datasets required thoughtful cross-validation strategies.
Accomplishments that we're proud of
High Accuracy Achievement: Reached 85%+ accuracy on benchmark datasets, outperforming many existing approaches through our multimodal fusion strategy.
Real-time Performance: Built a system that processes reviews instantly, making it practical for real-world deployment on high-traffic platforms.
User-Friendly Interface: Created an intuitive Streamlit application that makes advanced AI accessible to non-technical users, with live training progress and clear result visualization.
Flexible Architecture: Designed a modular system that works with text-only or multimodal inputs, adapting to different use cases and data availability.
Production-Ready Features: Implemented model persistence, comprehensive evaluation metrics, and robust error handling - features essential for real-world deployment.
Cross-Domain Generalization: Demonstrated effectiveness across different review types (hotels, products, restaurants) showing the model's versatility.
What we learned
AI Model Selection Matters: DeBERTa-v3 significantly outperformed BERT and RoBERTa for this task, highlighting the importance of choosing the right transformer architecture for specific applications.
Multimodal is the Future: Combining text and visual signals improved accuracy by 8-12% over text-only approaches, proving that multimodal analysis is crucial for sophisticated deception detection.
Data Quality > Data Quantity: Carefully curated, balanced datasets produced better results than simply using larger amounts of noisy data. Quality preprocessing and labeling were game-changers.
User Experience in AI: Building an intuitive interface for AI tools is as important as the underlying algorithms. Real-time feedback and clear visualizations make complex models accessible.
Scalability Considerations: Designing for production from day one (model persistence, batch processing, memory optimization) saves significant refactoring time later.
Domain Adaptation: Review deception patterns vary across industries, requiring careful consideration of domain-specific features and training strategies.
What's next for DeceptiGuard
Immediate Roadmap:
- Batch Processing: Upload CSV files for bulk review analysis
- Model Variants: Support for different DeBERTa sizes (small, base, large) based on speed vs accuracy needs
- Enhanced Image Analysis: Advanced computer vision techniques including attention mechanisms
- API Development: RESTful API for easy integration with existing platforms
Advanced Features:
- Multilingual Support: Expand beyond English to support global e-commerce platforms
- Temporal Analysis: Track review patterns over time to detect coordinated fake review campaigns
- Explainable AI: Provide detailed explanations of why reviews are flagged as deceptive
- Real-time Monitoring: Dashboard for continuous monitoring of review authenticity on platforms
Log in or sign up for Devpost to join the conversation.