Inspiration
We were inspired by how infrastructure deteriorates over time but inspection is manual and inconsistent. We wanted to automate visual change detection using AI.
What we learned
We learned about Siamese neural networks and transformer-based change detection models like ChangeFormer.
We also explored self-supervised techniques to reduce labeling effort.
How we built it
We used the LEVIR-CD+ and OSCD datasets, trained a Siamese U-Net, and evaluated using IoU and F1 metrics.
The model minimizes:
$$
\mathcal{L} = \text{FocalLoss} + \text{DiceLoss}
$$
to handle class imbalance in change regions.
Challenges faced
- Aligning “before” and “after” images (registration errors).
- Handling lighting and seasonal variations.
- Balancing limited labeled data with model accuracy.
Visual Difference Engine: Transfer Learning-Based Change Detection for Industrial Quality Assurance
- Problem Context & Impact Manufacturing industries face a critical quality control challenge: manual visual inspection is slow, expensive, and error-prone. Production lines generating thousands of components daily require human inspectors to identify defects like micro-cracks, surface anomalies, or assembly errors - a process where even experts miss 15-20% of defects due to fatigue. This leads to: ₹30-50 crore annual losses per manufacturing facility from defective products reaching customers Production bottlenecks as manual inspection cannot scale with increased output Inconsistent quality standards across shifts and facilities due to human variability Current automated inspection systems are domain-specific, requiring complete model retraining and thousands of labeled images for each new product type - making them impractical for manufacturers with diverse product portfolios. The industry urgently needs a general-purpose visual comparison engine that detects and classifies changes across time-series images with minimal adaptation effort.
Proposed Solution: Visual Difference Engine (VDE) We propose an AI-powered Visual Difference Engine using transfer learning with Siamese neural networks to detect and classify visual changes between image pairs captured at different time points. Our solution addresses the core challenge: building a production-ready change detection system within 24-hour hackathon constraints while maintaining industrial-grade accuracy. Core Innovation: Unlike conventional approaches requiring thousands of labeled examples per product type, our VDE leverages pre-trained vision foundation models (DINOv2 or EfficientNet-B4) that have learned universal visual features from 142 million images. Through transfer learning, the system adapts to new manufacturing scenarios with just 200-500 fine-tuning images - a 95% reduction in data requirements.
Technical Architecture (Research-Backed) Siamese Network with Transfer Learning Backbone: Our architecture employs a Siamese Swin-Unet configuration where two identical encoder branches (with shared weights) process "before" and "after" images simultaneously. This design has achieved 94.67% F1-score on benchmark change detection datasets. Research Foundation: Encoder: DINOv2 foundation model (Oquab et al., 2023) - a self-supervised vision transformer trained on diverse visual data, providing robust features that generalize across domains Architecture: Siamese Transformer architecture with U-shape decoder (Tang et al., 2024, Nature Scientific Reports) for pixel-level change segmentation Attention Mechanism: Full-image cross-attention (Lin et al., 2024) to handle viewpoint variations and lighting changes robustly Training Strategy (Zhao et al., 2023): Freeze pre-trained DINOv2 encoder layers Train only custom U-Net decoder and classification head on synthetic dataset Fine-tune last encoder layers for domain adaptation (Hours 10-12) This parameter-efficient approach reduces training time by 70% while improving accuracy by 10-15% compared to training from scratch - critical for hackathon success. Preprocessing Pipeline: Image Alignment: OpenCV-based feature matching and homography transformation to compensate for camera position variations (±5° rotation, ±10% translation) Lighting Normalization: Adaptive histogram equalization and CLAHE to handle illumination changes across time-series captures Change Classification: Beyond binary change detection, our system provides semantic classification into actionable categories: Surface defects (scratches, dents, corrosion) Missing/extra components Dimensional deviations Assembly errors Color/finish anomalies Each detected change includes: bounding box coordinates, confidence score, semantic label, and visual heatmap overlay.
Dataset Generation Strategy Data Sources We will utilize a small curated real-world dataset collected from open industrial quality control repositories (such as MVTec AD, DAGM 2007, and NEU-CLS) that contain authentic examples of surface defects, missing components, and assembly anomalies. These datasets provide representative defect types encountered across manufacturing domains, ensuring strong generalization. Data Preparation Pipeline Pair Formation: For each product, we construct before–after image pairs representing the defect-free and defective states. Automatic Alignment: Using OpenCV-based homography and SIFT feature matching, we ensure spatial consistency between image pairs to isolate true manufacturing changes. Lighting Normalisation: Adaptive histogram equalisation (CLAHE) and color constancy correction mitigate brightness or shadow differences caused by variable capture environments. Patch Extraction: High-resolution images are split into smaller overlapping patches (256×256 or 512×512) to balance training efficiency and maintain local defect visibility. Data Balancing: Oversampling of rare defect categories (e.g., corrosion or missing components) ensures balanced class representation without synthetic generation. Fine-Tuning & Adaptation For new product lines: We fine-tune the pre-trained DINOv2 encoder and U-Net decoder on 200–500 manually verified pairs captured from the production line. Domain adaptation is achieved through few-shot transfer learning, minimizing the need for large labeled datasets. This approach enables VDE to remain grounded in real industrial data, while achieving scalability and domain transferability—ideal for rapid hackathon deployment and long-term production use.
Research papers
https://arxiv.org/pdf/2312.04869 https://arxiv.org/pdf/2509.01754 Architecture(research backed): https://arxiv.org/html/2406.17998v1
Built With
- dl
- lstm
- python
- tranformers
- u-net
