plant disease classification

Inspiration

Plant diseases can spread quickly and cause significant losses, especially when farmers or plant owners do not have easy access to agricultural experts. We wanted to build a simple tool that could help people identify a plant and detect potential diseases using something they already have: a smartphone camera.

This idea led us to create an AI-powered system that analyzes a single image of a plant and provides information about the plant's identity and possible health condition.

How We Built It

Our project combines three specialized AI models working together, rather than relying on a single general-purpose model.

First, a DINOv2 vision transformer, fine-tuned on the PlantCLEF dataset covering 7,806 wild plant species native to south-western Europe, identifies the plant species from the uploaded image. Second, a dedicated disease classification model, trained specifically on common crop diseases (apple, tomato, corn), checks the same photo for visual signs of disease. Finally, a generative AI layer (Gemini) reviews both results alongside the original image to produce a clear, structured explanation covering the plant's overview, health assessment, diagnosis, treatment, and prevention tips — in language a non-expert can understand.

This layered design also lets the system catch its own mistakes: since the species model has a defined scope, the generative layer can flag and correct cases where the visual evidence contradicts the initial classification, rather than presenting a single unchecked answer.

We focused on making the experience straightforward: take a picture → analyze it → receive useful plant health information.

What We Learned

Building this project taught us that developing an AI application is more than simply connecting a model to an interface. The quality of the input image, the diversity of plant appearances, lighting conditions, backgrounds, and disease symptoms can all affect the reliability of the results.

We also learned that no single model can do everything well — combining specialized models, each with a clearly defined scope, produced more honest and reliable results than forcing one general model to cover every case. Just as importantly, we learned how critical it is to design AI outputs with the end user in mind. A technically impressive model is not enough if its results are difficult to understand or use.

Most importantly, we gained practical experience combining computer vision, generative AI, application development, and user-focused design into one working project.

Challenges We Faced

One of our biggest challenges was dealing with the natural variation in plant images. The same plant can look very different depending on lighting, camera quality, growth stage, background, and the severity of a disease.

Another challenge was that our disease detection model was trained on a limited set of common crops, while our species identification model covers thousands of species. Communicating this mismatch clearly to users — rather than hiding it — became an important design decision.

We also had to think carefully about how to communicate AI-generated results. Plant disease detection should be treated as an assistance tool rather than a replacement for professional agricultural diagnosis, so presenting the results responsibly, including their limitations, was an important part of the development process.

The Goal

Our goal is to make plant health analysis more accessible, fast, and easy to use. By combining specialized computer vision models with a generative AI layer that explains and cross-checks their results, we hope our project can help users recognize plants and identify potential diseases earlier, turning a simple photograph into clear, actionable information.

Built With

• dinov2
• gemini
• python
• pytorch
• scikit-learn
• streamlit
• timm
• transformers