Inspiration
After working on projects like detecting 14 diseases from a single X-ray (95.75% accuracy), building vision encoders for text-based LLMs(GPT-OSS), and predicting crop yields using satellite imagery, I realized a common gap — machines don’t truly understand visual change over time.
From brand labels slowly fading to cracks forming on a bridge, industries still rely on human eyes to spot differences that AI often misses.
MultiSense was born to close that gap — inspired by the need for a general-purpose Visual Difference Engine that can see, compare, and explain changes across time-series images with human-like intelligence
What it does
MultiSense is an AI-powered visual comparison engine that detects, classifies, and explains visual changes between images captured over time.
It automatically: -Highlights what changed (e.g., cracks, corrosion, missing components, color fade). -Classifies the type and severity of change. -Generates heatmaps, reports, and compliance scores.
Applications include manufacturing inspections, brand compliance, infrastructure degradation, and regulatory audits.
MultiSense doesn’t just show what changed , it explains why it matters.
How we built it
We designed a modular AI pipeline using Vision Transformers (ViT) and Siamese CNNs to combine classical and deep learning approaches:
1. Preprocessing & Alignment: OpenCV-based registration and adaptive brightness correction for time- series image alignment.
2. Dataset Preparation: -We plan to leverage a combination of public datasets (COCO, OpenImages, and infrastructure inspection collections) for pretraining.
-Synthetic augmentations (e.g., simulated cracks, fading, missing parts) will expand the dataset to improve model robustness and generalization.
3. Difference Extraction: Combined SSIM, MSE, and deep feature embeddings for robust detection.
4. Change Classification: ViT backbone identifies semantic-level differences (e.g., rust forming, paint fading).
5. Explainability: Attention and Grad-CAM heatmaps help users interpret AI outputs.
6.Visualization Layer: Streamlit dashboard generates real-time analytics, compliance heatmaps, and PDF summaries.
7.Architecture: Python-based microservices, Dockerized for scalability, with edge deployment support
Challenges we ran into
-Handling multi-domain variability - industrial, structural, and product images have very different patterns.
-Building trustworthy AI that not only detects but justifies its decisions.
-Maintaining real-time performance with high-resolution data.
-Designing a general-purpose model adaptable to diverse inspection environments.
Accomplishments that we're proud of
-Designed the architecture and workflow for a **universal visual comparison system **that can adapt to manufacturing, compliance, and infrastructure use cases.
-Created working prototypes for the AI pipeline and explainability layer using ViT and Siamese CNNs.
-Validated early models on time-series image datasets with promising results (~95% accuracy).
-Developed a scalable, modular, and edge-compatible architecture ready for real-world deployment.
What we learned
-How to combine transformer reasoning with classical vision metrics for better performance and interpretability.
-That context matters more than pixels — understanding what changed is more powerful than where.
-The importance of explainable AI for compliance and safety-critical tasks.
-True general-purpose AI requires cross-domain thinking and flexible architectures.
What's next for MultiSense
-Integrate with LLMs to generate natural-language “change explanations.”
-Extend to 3D and live video comparisons for continuous inspection.
-Develop MultiSense Edge, optimized for low-power industrial cameras.
-Partner with industries for real-world pilot testing and feedback.
-Release an open dataset for visual difference detection research.
Impact Metrics
Expected Impact:
-Reduce manual inspection effort by 70%
-Improve detection consistency by 35%
-Shorten compliance verification time by 50%
Small changes today can prevent big problems tomorrow — MultiSense makes them visible.
