Crop Guard

Plant diseases in agriculture cost the global economy $220 billion/year. We propose a multimodal AI approach to diagnosing crop diseases through an intuitive interface - empowering farmers of today 🌳

Comment

Inspiration

Crop diseases in agriculture cost the world ~$220 billion annually. Crop disease management tools are essential for efficiently addressing crop threats through targeted treatments, such as pesticides and herbicides.

What it does

Web application through which images can be uploaded. After an image of a crop is uploaded we leverage two models: a vision transformer model for disease classification and Gemini endpoint for offering information about the disease and how to address it. There is a login feature that allows users to access individualized analytics dashboards about their crop diseases.

How we built it

Frontend web - Next.js, Tailwind CSS, TypeScript Backend - Python, PyTorch, SQLite, Google Gemini, FastAPI Modeling: Finetuned 2 pre-trained ML models for disease classification: Microsoft's swin transformer and ResNet18. Final product integrated vision transformer model as well as Gemini API.

Challenges we ran into

Frontend: Faced many CSS bugs and React issues.

Backend: AWS RAM too small to allow pip install torch. Had issues storing images in SQLite database.

Modeling: class balance in dataset were initially skewing results on unseen images, requiring duplication of instances of minority classes.

Accomplishments that we're proud of

Implementing the individualized dashboard feature was important because it offers greater insight into the overall disease problems a user might have among their crops, which would be vital in a real-world setting.

What we learned

For Full Stack, we learned a lot about databases. We were trying to store images and had to dive deep into how they are represented as byte_values in order to embed them into a json response. We also learned a lot about AWS, when seeking deployment for our backend.

What's next for Crop Guard

The modeling would be improved by expanding the number of diseases that our model can predict as well as raising the accuracy using hyperparameter tuning (currently ~89%). The web application would be expanded to have the backend deployed on en EC2 instance as well as further data analysis and action-based insights on the user dashboards.

Built With

Share this project:

Updates