Inspiration Globally, up to 40% of crop yields are lost to pests and diseases annually, translating to millions of tons of wasted food. Traditionally, farmers combat this using blanket pesticide applications across entire plantations. This outdated method incurs heavy financial costs and severe environmental degradation, including soil toxicity and chemical runoff.
This global issue hit close to home while cultivating container-grown tropical fruit trees—like mangoes and lemons—and battling local pest populations. Instead of resorting to broad-spectrum chemicals, the goal was to manage pests using homemade, organic insecticidal potassium soap. We realized that by shifting to targeted spot-treatments, growers could significantly reduce environmental impact while preserving crop yields. AgroShield was born out of the need to automate this precision Integrated Pest Management (IPM) using accessible smartphone technology.
What it does AgroShield is a mobile-optimized web application that brings advanced computer vision directly to the field. A grower simply opens the app on their smartphone, points the camera at the underside of a leaf, and captures an image.
The application instantly analyzes the photo to detect and count whiteflies. Based on the severity of the infestation, it classifies the zone into a 3-tier risk system (Safe, Moderate, Critical). Instead of just giving a pest count, AgroShield prescribes a specific agronomic action—such as instructing the user to apply a spot-treatment of potassium soap. Every scan is automatically logged into a field audit database with exact coordinates (Zone, Row, Tree ID), allowing farmers to map infestation hotspots over time.
How we built it We engineered AgroShield as a full-stack application. The backend is powered by FastAPI (Python), which handles the image processing routes, model inference, and data persistence using SQLite.
For the frontend, we built a responsive, mobile-first dashboard using HTML5, Vanilla JavaScript, and Tailwind CSS. This ensures the interface is lightweight and integrates seamlessly with a mobile device's rear camera via the getUserMedia API.
The core AI engine is a custom-trained YOLO11 Nano object detection model. We curated a custom dataset of leaf undersides, annotated the whiteflies using Roboflow, and applied preprocessing techniques like image rotation and brightness adjustments to simulate varying canopy lighting. To ensure rapid and efficient training, we utilized a cloud-based Google Colab T4 GPU, which generated the optimized best.pt weights deployed in our final local application.
Challenges we ran into Building a complete machine learning pipeline and integrating it with a web application presented several hurdles:
Hardware Bottlenecks: Initially, we planned to train the computer vision model locally on an RTX 3050 laptop GPU, but VRAM limitations and thermal throttling slowed our progress. Pivoting the training pipeline to a cloud T4 GPU resolved this, but required dynamic file handling to pull the trained weights back to the local environment.
Mobile Camera Integration: Securely connecting a mobile device (like a Pixel phone) to a local development server for live camera testing required navigating HTTPS restrictions and remote USB debugging via Chrome's inspection tools.
Dataset Structuring: Configuring the data.yaml paths correctly and ensuring Roboflow smoothly split our custom yard photos into the correct training and validation directories took careful iteration.
Accomplishments that we're proud of We are incredibly proud of bridging the gap between software engineering and agricultural science within the timeframe of a hackathon. Successfully training a custom YOLO11 model that accurately draws bounding boxes around tiny whiteflies in real-world, varying lighting conditions is a major milestone. Furthermore, developing a zero-dependency fallback "Simulation Mode" ensures the app remains testable and functional even if the machine learning weights are temporarily offline.
What we learned This project was a deep dive into the practical applications of AI in agriculture. We learned how to architect a complete YOLO-based machine learning pipeline—from manual image annotation in Roboflow to configuring epochs and batch sizes for optimal training. We also gained highly practical experience in connecting Python-based FastAPI backends with asynchronous JavaScript frontends, and dealing with device-level hardware permissions for web applications.
What's next for AgroShield The next step is expanding the model's intelligence. We plan to enrich the dataset to detect a wider variety of pests (such as aphids and spider mites) as well as beneficial insects (like ladybugs) to provide a more holistic view of the ecosystem. We also aim to deploy the application to a live cloud environment to eliminate the need for local server hosting, and introduce a heat-mapping feature that visualizes SQLite field data over a satellite view of the user's farm.
Log in or sign up for Devpost to join the conversation.