Inspiration
What## Inspiration
Industrial defect inspection is essential for modern manufacturing, but traditional supervised methods usually require large amounts of pixel-level annotated data. In real production environments, defect samples are rare, diverse, and expensive to label.
This project was inspired by AnomalyCLIP, which transfers the general visual-language knowledge of CLIP to zero-shot anomaly detection. However, we found that the original model may struggle with small defects, multi-scale anomalies, noisy prediction regions, and unclear defect boundaries.
What It Does
AnomalyFusion is an enhanced industrial anomaly detection framework based on AnomalyCLIP. It focuses on improving both image-level defect recognition and pixel-level anomaly localization.
The framework introduces two main components:
- Adaptive Spatial Feature Fusion (ASFF): combines multi-stage visual features and automatically adjusts the contribution of different feature scales.
- Residual Refinement Module (RRM): learns residual corrections for the initial anomaly map, helping suppress background noise and recover more accurate defect boundaries.
The model produces:
- An image-level anomaly score
- A pixel-level anomaly heatmap
- A visualized defect localization result
How We Built It
The system is implemented with Python and PyTorch. We use the CLIP ViT image encoder to extract patch-level features from multiple network stages.
These visual features are aligned with learnable normal and abnormal text representations. ASFF then combines features from different semantic levels, while the lightweight RRM refines the coarse anomaly prediction.
The project is evaluated on industrial anomaly detection datasets such as MVTec AD and VisA, using metrics including image-level AUROC, pixel-level AUROC, AUPRO, and F1 score.
Challenges
One major challenge was improving defect localization without damaging the original zero-shot generalization capability.
Different defects also appear at significantly different scales. Small scratches require fine-grained shallow features, while larger structural defects depend more on deep semantic features. Balancing these features required repeated experiments and ablation studies.
Another challenge was controlling the refinement module. Excessive refinement may improve visual boundaries while introducing false-positive regions, so the module must remain lightweight and carefully supervised.
What We Learned
We learned that higher model complexity does not always lead to better anomaly detection performance. Multi-scale feature fusion and anomaly-map refinement must be evaluated separately through controlled ablation experiments.
We also learned the importance of:
- Running experiments with multiple random seeds
- Reporting both image-level and pixel-level metrics
- Comparing model accuracy, stability, and computational cost
- Avoiding conclusions based on a single experimental result
What's Next
The next steps are to complete multi-seed experiments, conduct full ablation studies, evaluate cross-dataset generalization, and further reduce the computational cost of the refinement module.
We also plan to develop a simple visual interface that allows users to upload an industrial image and obtain an anomaly score and defect heatmap. it does
Log in or sign up for Devpost to join the conversation.