-
-
Home page
-
Scanning page
-
Performance Summary
-
Result page
-
Result page
Inspiration
🌿 AgriSight: Explainable Early Crop Stress Detection
Catch crop stress before it shows — with transparent AI you can trust in the field.
💡 Inspiration
Agriculture is the backbone of global food security, yet smallholder farmers often lose 20–40% of their annual crop yield to preventable diseases and pest stress. While modern computer vision promises automated diagnostics through a simple smartphone photo, there is an uncomfortable industry secret: models that achieve 98%+ accuracy on clean, lab-bench datasets fail dramatically when deployed in real, messy agricultural fields.
Most existing apps deliver confident, silent wrong predictions when confronted with harsh sunlight, cluttered soil backgrounds, or multiple leaves. We were inspired to build AgriSight not just to detect crop diseases, but to bring uncompromising transparency and explainability to agronomic AI. We wanted to answer two vital questions:
- How large is the real-world domain gap between laboratory-trained models and chaotic field conditions?
- Can field-style domain augmentation and visual explainability (Grad-CAM) protect farmers from misleading automated diagnoses?
🌾 What It Does
AgriSight is a full-featured, farmer-first diagnostic portal that detects 18 crop health conditions across staple crops (Tomato, Potato, Corn, Apple, Grape, and Bell Pepper).
Key capabilities include:
- Instant Leaf Scan & Multi-Model Inference: Classifies uploaded or camera-captured leaf images using state-of-the-art backbones (MobileNetV3-Large, EfficientNet-B0, and Baseline CNN).
- Grad-CAM Attention Heatmaps: Overlays visual activation heatmaps showing exactly which pixel regions influenced the model's decision, ensuring the model focuses on lesion pathology rather than soil, stems, or background artifacts.
- Border-Attention & Anomaly Detection: Implements automated heuristics that alert users if attention concentrates on the image border ($>80\%$ attention in outer $20\%$ margins), catching spurious background correlations before they misinform.
- Honest Confidence Thresholds: Employs an adjustable confidence gate ($60\%$ baseline). Low-confidence scans do not provide a guessed disease; instead, they transparently advise the farmer on taking a clearer, well-lit photo.
- Interactive "Model Trust" Dashboard: Directly displays real-world validation metrics, confusion matrices, and robustness benchmarks under simulated adverse conditions (dim light, blur, high noise).
- Farm Record Management: Allows farmers to save scan histories with custom field notes, track disease recurrence, and export printable PDF diagnostic summaries.
🛠️ How We Built It
1. Dataset Harmonization & The Real-World Domain Benchmark
We benchmarked our models across two distinct visual domains:
- Source Domain (Lab / Controlled): The PlantVillage dataset (clean leaves on uniform, isolated backgrounds).
- Target Domain (Field / In-The-Wild): The PlantDoc dataset (leaves photographed under real farm canopies with natural light variation, complex backgrounds, and multiple overlapping foliage).
We mapped and harmonized both datasets to 18 identical shared classes, filtering PlantVillage to $\le 600$ images per class ($6,882$ training images total) and restructuring PlantDoc's YOLO bounding-box annotations into an unseen test benchmark ($4,145$ real-field images).
2. Training Pipeline & Mathematical Objective
We implemented a two-stage transfer learning strategy:
- Stage 1 (Feature Extraction): Backbone frozen; training classification head for 5 epochs with AdamW ($\text{LR} = 10^{-3}$).
- Stage 2 (Fine-Tuning): Unfreezing top backbone layers with reduced learning rate ($\text{LR} = 10^{-4}$, cosine annealing decay).
To address class imbalance and overconfident mispredictions, we utilized class-weighted cross-entropy loss with label smoothing ($\alpha = 0.05$): $$ \mathcal{L}{\text{LS}}(y, \hat{y}) = - \sum{k=1}^{K} \left( (1 - \alpha) y_k + \frac{\alpha}{K} \right) \log(\hat{y}_k) $$
3. Field-Style Domain Augmentation
To bridge the visual gap without collecting thousands of new field images, we developed custom field-style augmentations mimicking outdoor conditions: strong color jitter, Gaussian blur, random perspective distortion, rotation, and CutOut / Random Erasing.
We defined and tracked the Domain Gap Metric: $$ \text{Domain Gap} = \text{Accuracy}{\text{PlantVillage (Lab)}} - \text{Accuracy}{\text{PlantDoc (Field)}} $$
4. Explainable AI & Grad-CAM Integration
We extract activation maps from the final convolutional layers ($A^k$) and weight them by class-specific gradients: $$ L_{\text{Grad-CAM}}^c = \text{ReLU}\left(\sum_{k} \alpha_k^c A^k\right) \quad \text{where} \quad \alpha_k^c = \frac{1}{Z}\sum_{i}\sum_{j} \frac{\partial Y^c}{\partial A_{i,j}^k} $$
The user interface was built using Streamlit, featuring clean visual hierarchy, guest mode, optional Supabase persistence, and mobile-friendly layouts.
📊 Results & Scientific Findings
| Model | Augmentation | Lab (PlantVillage) Acc | Field (PlantDoc) Acc | Domain Gap | Blur Acc | Dim-Light Acc | Noise Acc |
|---|---|---|---|---|---|---|---|
| Baseline CNN | Basic | 90.9% | 6.3% | 0.845 | 27.7% | 19.8% | 45.5% |
| EfficientNet-B0 | Field | 97.7% | 14.5% | 0.832 | 81.2% | 97.5% | 76.7% |
| MobileNetV3 | Basic | 98.6% | 12.1% | 0.865 | 27.7% | 96.3% | 83.6% |
| MobileNetV3 | Field | 97.8% | 16.0% | 0.819 ✅ | 77.8% | 97.3% | 88.4% |
Key Takeaways:
- The Domain Gap is Real: Despite achieving $>98\%$ lab accuracy, standard models drop precipitously when faced with uncurated field imagery.
- Field Augmentation Closes the Gap: Applying field-style augmentation narrowed the domain gap from $0.865$ to $0.819$ (+5% generalization gain) and increased blur robustness from $27.7\%$ to $77.8\%$.
🧗 Challenges We Faced
- The Shortcut Learning Problem: Early Grad-CAM tests revealed models were learning to associate lab background cardboard colors with particular disease classes. Introducing aggressive background erasing and border checks was essential to eliminate background shortcuts.
- PlantDoc Annotation Reorganization: PlantDoc was distributed primarily as raw bounding-box annotations. We had to build an automated pipeline to crop coordinates, map disparate taxonomy labels to match PlantVillage names, and handle multi-leaf overlaps.
- Balancing Model Size with Farmer Usability: Complex vision transformers are too heavy for remote rural deployment. Optimizing lightweight backbones (MobileNetV3 and EfficientNet-B0) allowed us to achieve sub-second inference while retaining high representational power.
🏆 Accomplishments We're Proud Of
- Honest Research Over Vanity Metrics: Rather than claiming $99\%$ accuracy, we explicitly publish our field domain gap and failure cases in the UI so users know the technology's real limits.
- Actionable Explainability: Our Grad-CAM and Border-Attention heuristics actively safeguard farmers from silent hallucinations.
- Zero-Barrier Access: Anyone can immediately evaluate leaves in Guest Mode without needing to register or configure an account.
📚 What We Learned
- A high test-set score on benchmark lab datasets is fundamentally insufficient for safety-critical AI tasks like food production.
- Explainability isn't just a post-hoc inspection tool for engineers — it is an active UI safety feature for the end-user.
- Augmentations targeted directly at physical domain discrepancies (lighting, focus, camera angles) provide significant generalization gains without costly data re-collection.
🚀 What's Next for AgriSight
- On-Device Edge Deployment: Quantizing the MobileNetV3 weights into ONNX / TFLite for completely offline mobile phone inference in remote areas with zero network connectivity.
- Multi-Leaf Bounding Box Detection: Upgrading from single-leaf classification to full-canopy YOLO detection for field-scale scanning.
- Agronomic Treatment Recommender: Integrating localized agronomy advisories and organic remedy steps tailored to regional soil and climatic factors.
Log in or sign up for Devpost to join the conversation.