About the project

Inspiration

Agriculture and forestry are critical for food security, climate resilience, and biodiversity. Yet, monitoring crops and trees at scale remains a challenge. I’ve always been fascinated by satellites, drones, and geospatial technology, and I wanted to build something that could see from above and make sense of the landscape. This project was born out of that passion and the desire to apply deep learning to real-world environmental challenges.

What it does

FarmSight AI is a vision-based system that classifies vegetation species from aerial imagery. In our prototype, we focused on tree species classification from UAV images by leveraging multi-angle inputs. Instead of relying only on top-down views, the model fuses nadir (90°) and oblique (60°) perspectives, enabling it to capture both texture and structural cues of vegetation. This approach improves accuracy, robustness, and generalization across different viewing conditions.

How we built it

• Preprocessed UAV RGB images (blur filtering, normalization, class balancing).
  
• Trained view-specific CNNs:
  
  • ConvNeXt & EfficientNetV2S for nadir imagery.
    
  • ResNet50V2 & DenseNet121 for oblique imagery.
    
• Designed a feature-level fusion model combining EfficientNetV2S and DenseNet121 representations.
  
• Implemented with TensorFlow/Keras, fine-tuned on the Swedish Forest Agency dataset.
  
• Achieved ~90% accuracy and F1-score (0.90) on the combined multi-angle dataset.
  

Challenges we ran into

• Domain gap: models trained on one view (nadir) dropped performance on another (oblique).
  
• Data imbalance: minority classes like Aspen were underrepresented, causing instability.
  
• Computation: training multiple deep models and fusion required efficient GPU usage.
  
• Generalization: ensuring robustness across angles and flight paths was non-trivial.
  

Accomplishments that we're proud of

• Built a fusion-based deep learning pipeline that closes the performance gap between nadir and oblique views.
  
• Achieved strong, balanced performance across species with an F1-score of 0.90.
  
• Demonstrated that multi-angle integration significantly outperforms single-view baselines.
  
• Took a step closer to scalable, real-world applications in agriculture and forest monitoring.
  

What we learned

• The importance of cross-domain generalization in remote sensing.
  
• How multi-angle perspectives can complement each other to improve AI accuracy.
  
• Advanced CNN architectures and how to fine-tune them effectively.
  
• Practical aspects of data preprocessing (noise removal, normalization, balancing).
  
• That AI for environmental monitoring can be both technically challenging and deeply impactful.
  

What's next for FarmSight AI

The next step is to extend this approach from UAV imagery to satellite datasets (Sentinel-1/2, ResourceSAT, AgriFieldNet India). By scaling to satellite data, FarmSight AI can help:

• Classify crops across fragmented farmlands.
  
• Support farmers with better advisory and insurance insights.
  
• Assist policymakers in food security and resource planning.
  This project is an exciting foundation, and the hackathon is the perfect opportunity to take it from UAVs to large-scale, satellite-driven agriculture solutions.
  

Built With

• convnext
• densenet121
• efficientnetv2s
• fusion-cnn
• fusion-cnn-libraries:-numpy
• github
• jupyter-notebook
• kaggle-notebook
• keras
• keras-models/architectures:-convnext
• matplotlib
• numpy
• pandas
• python
• resnet50v2
• scikit-learn
• scikit-learn-tools:-jupyter-notebook
• tensorflow