Multi-Modal Deep Learning Model for Crop Classification

Abstract: The Agricultural Sector forms the backbone of the Indian Economy, contributing significantly to GDP and employment. It is uniquely challenged with respect to cropping practices and patterns due to most farms being fragmented and small(< 2 hectares), making homogeneity difficult to achieve.Also,the spatial resolution of available satellite imagery often results in a single pixel covering multiple fields or topographical features, making crop identification difficult, and hence the prediction accuracy drastically reduces.This project proposes a Deep Learning based solution implementing Geospatial Foundation Models trained on data from Satellites like Sentinel-1,Sentinel-2 and ResourceSAT-2/2A. This project aims to incorporate feature extraction from the AgriFieldNet dataset, and also recognize patterns to develop a transformer-based multimodal architecture while also analysing time series data using multi-temporal SAR to accurately classify various crops.The aim is to improve classification accuracy for major crops while addressing imbalance issues present in Indian agriculture. The system developed will provide real-time, ground level crop monitoring capabilities and will be extremely useful for Food Security, Policy Formation applications, while giving crucial insights on improvements to Farming Techniques, thus enhancing the Indian Agricultural Landscape.

Built With

• amazon-web-services
• docker
• fastapi
• mlflow
• python