KhudAI

Problem statement

Existing models for crop classification fail to generalize across the fragmented and highly diverse agricultural landscape of Northern India. The core technical problem lies in the ineffective fusion of SAR and optical satellite data.

Simple approaches cannot combine the structural information from SAR (which penetrates clouds) with the rich spectral detail from optical sensors. This leads to poor performance, a lack of robustness, and unreliable predictions. Our work aims to directly confront this critical gap in multi-modal data fusion, which we believe is the primary obstacle to deploying a truly scalable and accurate crop monitoring system for the region.

Proposed solution

We propose a novel, resource-efficient AI pipeline that fuses state-of-the-art architectures to address these challenges. Our core strategy involves parameter-efficient fine-tuning of IBM's Prithvi, a large geospatial foundation model, using Low-Rank Adaptation (LoRA) to transfer powerful pre-trained knowledge with minimal computational overhead.

This fine-tuned model will integrate a hybrid CNN-Transformer architecture capturing fine-grained local textures and broader field context. A hyperspectral-inspired spectral attention mechanism will guide the model to focus on the most informative spectral bands (e.g., near-infrared, red-edge) for distinguishing key Indian crops. Additionally, we plan to integrate a synchronised class token fusion mechanism (SCTF) within the Transformer to explicitly enrich class representation and balance learning across dominant and rare crop categories.

Class imbalance will be further managed by weighted loss functions and targeted data augmentation. SCTF enhances the model’s ability to jointly learn spatial and spectral features linked to crop classes, improving classification accuracy for less common crops. This results in a scalable, accurate tool to unlock precise crop classification for millions of smallholder farmers.