Local AI Plant Medic

• 

  Home

• 

  Analysis

Inspiration

Local Plant Medic was inspired by UN Sustainable Development Goal 2: Zero Hunger. In many rural farming communities and isolated home gardens, internet connectivity is spotty or nonexistent. When crop diseases strike, waiting for an expert or an internet connection can lead to massive food loss. I wanted to build a fast, reliable tool that empowers farmers to diagnose plant health issues immediately on-device—no Wi-Fi or cellular connection required for the Android version.

What it does

Local Plant Medic (also known as Flora Diagnostics) is a fully offline application that turns a mobile device or browser into a plant doctor. Users can snap or upload a photo of a plant leaf, filter by crop type (such as wheat or rice), and receive instant locally processed diagnostics. The app provides:

Condition Analysis: Percentage-based confidence scores for suspected plant diseases.

Symptom Checkers: Diagnostic summaries outlining what is happening to the plant.

Recommended Treatments: Immediate, practical actionable advice to mitigate disease and prevent crop loss.

How we built it

Framework: I built the application using Next.js for a responsive, modern web interface.

Local Machine Learning: I integrated ONNX Runtime to run directly in the browser via worker.js. Image data is processed as Uint8Array buffers and wrapped in RawImage objects to execute model inferences completely offline without hitting external servers.

Mobile & Web: I deployed the web version on Vercel and added Android build files to package and generate offline APK artifacts via GitHub.

Challenges we ran into

ONNX & Data Formats: Passing raw image data smoothly to the local model required debugging low-level image pipeline logic, ensuring data was correctly converted to a Uint8Array and properly wrapped for the ONNX worker.

Prediction & Filtering Logic: Early iterations struggled with model bias—the logic eliminated lower-probability cases too aggressively, leading to empty result outputs for almost every crop except rice. I had to rewrite the thresholding logic to better process varied inputs like wheat.

Build & Cache Conflicts: Fixing Next.js configurations (next.config.ts), clearing cache, and stripping out breaking server commands were essential to achieving a clean, reliable build.

Accomplishments that we're proud of

100% Offline AI Inference: Successfully running plant diagnostic vision models locally using client-side Web Workers.

Multi-Platform Support: Expanding the initial web application into buildable Android APK binaries.

Actionable Medical Reports for Crops: Designing a UI that displays the disease name, probability confidence percentage, condition description, and recommended treatment.

What we learned

Client-Side Model Optimization: Working with ONNX models taught me the intricacies of memory management, Web Workers, and data structure formatting (RawImage and byte array buffers) in web environments.

Agriculture-Specific Edge Cases: Machine learning models require careful tuning for real-world field conditions—adjusting probability cutoffs is essential so useful diagnostic warnings aren't accidentally filtered out.

What's next for Local AI Plant Medic

Broader Crop Training: Expanding the dataset and local model training to cover a significantly wider range of regional agricultural crops, fruits, and vegetables.

Better Plant Scope Filtering: Refining pre-analysis plant selection to narrow the diagnostic scope and increase prediction accuracy.

Full Mobile Distribution: Perfecting the Android APK pipeline to easily deliver downloadable offline mobile packages for farmers in low-connectivity regions worldwide.

Built With

• android
• apk
• css
• javascript
• next
• next.js
• onnx