Labeled Camera Prototype
Colored Camera Prototype
Extendable Design Height Comparison
Dashboard
Pest Scan
Disease Scan
Soil Monitor
Sample Pest Scan Result
Sample Disease Scan Result
Sample Field Health Report
5-Year Plan Prototype Comparison
AURAfarming Feedback Form
AURAfarming Logo
AURAfarming Title Slide

Project Plan

Inspiration

I've always been interested in computer vision, but the idea for AURAfarming was sparked by another image classification project I saw a few years ago related to classifying crops and weeds. That was the first time I'd seen the tech applied to agriculture, which encouraged me to research the problems that affect farmers worldwide.

Three problems kept coming up everywhere I looked: pests, plant diseases, and soil degradation. The FAO puts crop losses from pests and disease alone at over $300 billion a year, and when I reached out directly to agricultural organizations in the US, Canada, Ireland, and India, every single one confirmed the same three concerns. AURAfarming was built with the goal to develop a unique solution for all three of these problems at once.

What it does

AURAfarming is a crop monitoring system with a hardware concept, CAMAURA v2, and a working software demonstration. The physical cameras continuously take photos of crops, and two computer vision models process the images. One scans for pests and draws a bounding box around each one it finds, the other classifies leaf photos for signs of disease. Both feed into a plain language risk assessment with a severity rating and treatment recommendation, so the farmer gets an answer and is able to take action accordingly. That data sits alongside a live soil health dashboard that tracks moisture, salinity, pH, nitrogen, and temperature, rolled into a single composite score on one simple interface, so a full picture of the field is easily visible instead of having to rely on three separate systems.

How it was built

The software runs as a local Flask (Python) app, with the frontend built using Jinja2 templates and plain HTML, CSS, and JavaScript rather than a frontend framework, so the whole thing stays lightweight and easy to run, even in a rural area without reliable internet.

Pest detection uses a Ultralytics YOLO11 computer vision model, pulled from the Hugging Face Hub, fine-tuned for pest identification. Photos are run through SAHI (Slicing Aided Hyper Inference), which tiles each image into overlapping slices, runs detection on each tile at near full resolution, and merges the results. This is what makes it possible to catch small or tightly clustered pests, like an aphid colony, that a single full frame pass tends to blur together or miss. OpenCV draws the bounding boxes and labels back onto the original photo.
Disease detection uses a MobileNetV2 image classifier from Hugging Face Transformers, fine-tuned on the PlantVillage dataset. Rather than relying on the transformers pipeline's built in label resolution, inference is run manually and class names are read straight from the model's own id2label config. A complementary OpenCV heuristic with HSV color thresholding, refined with GrabCut foreground segmentation, highlights the likely affected regions on the leaf as a visual aid alongside the model's diagnosis.
Soil monitoring uses a canvas line chart with no external charting library for the dashboard's trend view to keep the app dependency light for spotty rural connections. The current software demo simulates realistic sensor drift for moisture, salinity, pH, nitrogen, and temperature, standing in for the physical soil probes on CAMAURA v2 while hardware integration is in progress.

The full source is on GitHub, with more detail on the model pipeline and setup instructions if you want to run it yourself.

Challenges

Both models needed a lot of iterative tuning to get right. The pest detector would either be too conservative and miss real pests, or overcount by detecting the same pest more than once across overlapping tiles. Getting the confidence threshold right took a lot of trial and error against real test photos.

The disease model had two separate issues. First, diagnoses were coming back as Unknown no matter what was in the photo. It turned out the transformers pipeline's internal label lookup was silently falling back to placeholder labels instead of real class names, so nothing ever matched. Reading the label mapping directly off the model's own config fixed this issue. Additionally, the color based overlay that highlights affected leaf area was missing genuinely dark spots and sometimes flagging plain background as diseased. Both traced back to how the foreground and background segmentation was interacting with the darkness threshold, and both took several rounds of tuning to balance against each other without reintroducing the other problem.

Accomplishments

I'm especially proud of how accurate the pest scanner ended up. One of my main test images had 18 different pests on a single plant, many overlapping or partly hidden, which is a difficult case in the context of object detection. After tuning, the model landed on a count of 17, which validated that the threshold fine-tuning had paid off.

I also really love the PDF export feature. It generates a full field health report, pulling together pest activity, plant disease status, and soil health into one plain language summary. This tool could be especially useful if a farmer needs to send a quick report of their field to an agronomist or co-op, saving them from having to type up notes by hand.

What was learned

Before I built anything, I learned how important it is to validate a problem before trying to solve it. I reached out to agricultural organizations like SARE, the National Farmers Union, Teagasc, and FAIFA, and hearing directly from people in the industry that pests, disease, and soil degradation were real ongoing concerns for their farmers gave me confidence I was building toward something that actually mattered, not just a problem I assumed existed.

I also learned a lot about hardware design, which definitely isn't my strong suit. Building even a rough prototype of the CAMAURA camera housing was difficult, but creating a quick prototype as well as a feedback form for suggestions helped me make substantial improvements. The responses I received on the feedback form included questions about mounting and power, which gave me the idea to incorporate an extendable rod and solar panels in v2. This taught me that feedback from others is important to not only integrate suggestions, but to help generate your own ideas.

What's next for AURAfarming

The near term goal is building a working CAMAURA hardware prototype and wiring it into the existing software, so soil readings come from real sensors instead of a simulation.

Beyond that, the plan follows a 5 year path. The first few years involve R&D, prototype refinement, and initial government partnerships. Eventually, if the prototype was manufactured, the plan would be to target a stabilized deployment across roughly large-scale farms on multiple continents. Given that government agencies make up an estimated 83% of the addressable market, prioritizing that channel early is the fastest path to the break even point the model is built around. The full 5 year timeline can be found in the Project Plan, along with a budget breakdown and risk analysis.