Inspiration
A crop disease can become an environmental problem long before it looks like one.
I grew up around smallholder farmers in Uganda and repeatedly saw the same bottleneck: a farmer could see that a crop was unhealthy, but access to agricultural expertise was often distant, expensive, or delayed. By the time a problem was correctly identified, it could already have spread across a field.
That matters beyond one harvest. When farmers lose crops, they face pressure to compensate through additional cultivation, increased inputs, or other costly decisions. Early detection therefore has the potential to protect not only productivity, but the agricultural ecosystems that productivity depends on.
At the same time, many farmers already carry a powerful sensing device in their pockets: a smartphone camera.
That led to a simple question:
What if a farmer could photograph a sick plant and immediately receive an evidence-grounded first layer of agricultural intelligence?
AgriDocAI was built to explore that idea.
Rather than replacing agricultural extension workers or agronomists, AgriDocAI is designed as an accessible first layer of intelligence that can help identify problems earlier, explain what is happening, and support better decisions before a localized crop problem becomes a larger one.
What it does
AgriDocAI turns a smartphone photograph into an actionable agricultural assessment in seconds.
A farmer photographs a cassava, maize, banana, or coffee leaf. The system analyzes the image, identifies likely disease symptoms, explains the reasoning, suggests next steps, and incorporates local weather conditions into its recommendations.
The system is designed to be more than a simple image classifier.
Visual symptom grounding
AgriDocAI can identify regions of the farmer's own image that contributed to the diagnosis using numbered visual annotations.
This gives the user something to inspect rather than receiving an unexplained label.
Differential reasoning
The system can present alternative conditions considered during diagnosis rather than hiding uncertainty behind a single answer.
Uncertainty-aware diagnosis
Not every case deserves the same level of computational effort.
AgriDocAI uses a confidence-triggered cascade: uncertain cases can be escalated from Gemini Flash to Gemini Pro for additional reasoning instead of forcing the system to produce a low-confidence answer.
Weather-aware recommendations
Current district-level weather from Open-Meteo is incorporated into recommendations, connecting visual diagnosis with real environmental conditions.
Confidence calibration
A model-generated confidence percentage is not automatically reliable.
AgriDocAI includes infrastructure designed to compare predicted confidence against farmer- or expert-confirmed outcomes over time, creating a pathway toward measurable calibration rather than assuming that model confidence equals real-world accuracy.
Personalized learning
Each diagnosis can become a micro-lesson, allowing interactions to build a crop-specific learning history from the farmer's own cases.
Expert review
Cases can be exported as structured packages for agronomist validation, creating a human-review pathway for difficult cases.
Multilingual access
English, Luganda, and Swahili voice output are supported to make agricultural intelligence more accessible across language contexts.
Outbreak intelligence
With explicit opt-in, aggregated observations can eventually contribute to district-level disease surveillance.
The important distinction is that AgriDocAI does not fabricate environmental intelligence to make the product look more impressive. Sample outbreak information is clearly identified as sample data.
The long-term vision is to connect individual plant observations → local agricultural intelligence → broader environmental signals.
How we built it
AgriDocAI is a lightweight web application built with HTML, CSS, and JavaScript, designed around low-cost and low-infrastructure deployment.
The diagnostic pipeline is deliberately layered.
1. On-device computer vision
Before an AI request is made, the browser performs HSV colour-space analysis of the uploaded leaf image.
It estimates the proportion of visible tissue associated with:
- healthy green
- yellowing
- necrosis
This produces a local visual signal that can supplement multimodal reasoning.
2. Evidence-grounded multimodal AI
The visual information is combined with a curated reference set covering major documented diseases for the selected crops, based on Uganda-focused agricultural literature.
That context is passed with the image to Gemini Flash for multimodal reasoning.
Instead of treating the model as an isolated oracle, the architecture provides crop- and region-relevant reference information as grounding context.
3. Confidence-triggered model cascade
When an initial diagnosis falls below the defined confidence threshold, the system escalates the same case to Gemini Pro.
This creates a practical architecture:
fast reasoning → confidence assessment → deeper reasoning when required
rather than using the most expensive model for every case.
4. Environmental context
Open-Meteo supplies current district-level weather information.
That information is incorporated into recommendations so that the system considers environmental conditions alongside the visual evidence.
5. Case storage and longitudinal infrastructure
Firebase / Firestore provides anonymous authentication, case storage, outbreak-data infrastructure, and calibration infrastructure.
This allows diagnoses to become longitudinal observations rather than isolated interactions.
6. Resilient AI access
Puter.js provides a fallback AI access route when a direct Gemini API key is unavailable.
This supports deployment in environments where access to a single API pathway may not be guaranteed.
7. Explainability validation
Vision models do not always return reliable spatial coordinates.
AgriDocAI validates returned coordinates before rendering symptom annotations. When the coordinates are inconsistent with the expected output, the system falls back to a text-based explanation rather than displaying misleading visual evidence.
The architecture therefore treats reliability as a system-level problem, not simply a model-selection problem.
Challenges we ran into
The hardest part was not getting an AI model to produce a diagnosis.
The harder problem was designing a system that could be useful without pretending to know more than it actually knows.
Working within real infrastructure constraints
Our first approach relied heavily on a free AI proxy. Image-heavy usage exposed storage and reliability limitations.
We responded by redesigning the architecture around direct Gemini access with a fallback route instead of assuming unlimited infrastructure.
Making uncertainty meaningful
It is easy to display a confidence percentage.
It is much harder to determine whether the percentage deserves to be trusted.
That led us to build calibration infrastructure so that predicted confidence can eventually be compared with real outcomes confirmed by farmers or agricultural experts.
Avoiding fabricated environmental data
We considered making the outbreak map appear more populated using synthetic cross-user observations.
We deliberately rejected that approach.
A fake environmental signal can look harmless inside a demo but become dangerous if interpreted as evidence of real disease activity.
Sample data is therefore explicitly labeled as sample data.
Making visual explanations reliable
Multimodal models do not always provide dependable spatial coordinates.
We added validation around the annotation system and introduced a fallback to text-only explanations when the visual coordinates cannot be trusted.
This was a broader lesson:
an unreliable explanation is worse than no explanation.
Accomplishments that we're proud of
We are proud that AgriDocAI evolved beyond a conventional agricultural image-classification demo.
The current system combines:
on-device computer vision → evidence-grounded multimodal reasoning → uncertainty-triggered escalation → weather context → treatment guidance → visual explanation → expert review → confidence calibration infrastructure → potential outbreak intelligence
That is a complete decision-support pipeline rather than a single model call.
We also built the system as a solo developer using essentially zero-cost infrastructure.
But the accomplishment we value most is architectural rather than visual.
We deliberately chose to build safeguards around uncertainty instead of optimizing only for impressive demonstrations.
The system can escalate uncertain cases.
It can reject unreliable spatial annotations.
It can distinguish sample environmental data from real observations.
And it has infrastructure designed to measure whether its confidence means anything in the real world.
Our core principle became:
If the system does not know, it should not pretend that it does.
For an AI system intended to influence agricultural decisions, we believe that principle is part of the engineering—not just the messaging.
What we learned
The biggest lesson was that building an AI product is not the same thing as calling an AI model.
Our original concept involved using several models simultaneously and allowing them to vote on every diagnosis.
We replaced that with a confidence-triggered cascade.
The resulting architecture is faster and more cost-conscious while reserving deeper reasoning for the cases where uncertainty actually warrants it.
We also learned that calibration matters more than impressive-looking confidence numbers.
A system saying “95%” does not mean that it is correct 95% of the time.
Confidence only becomes meaningful when it is compared with outcomes.
Most importantly, we learned that responsible engineering sometimes makes a project look less impressive.
We could have filled the outbreak map with synthetic observations.
We did not.
We could have treated every model output as equally trustworthy.
We did not.
Instead, we built uncertainty handling, validation pathways, and explicit distinctions between real and sample data.
That changed how we think about environmental technology:
the goal is not simply to build technology that works in a demo. It is to build technology whose outputs can eventually be trusted in the environment where it operates.
What's next for AgriDocAI
The next step is formal validation.
We plan to evaluate AgriDocAI against labeled crop-disease datasets and, more importantly, build a collection of expert-reviewed field cases.
The evaluation will focus on:
diagnostic accuracy, confidence calibration, failure modes, crop-specific performance, image-quality robustness, and reliability of visual explanations.
Beyond validation, the roadmap includes expanding the disease reference set, supporting additional crops and districts, and developing SMS/USSD access for farmers with unreliable mobile data.
The longer-term opportunity is environmental intelligence.
With enough real, opt-in observations, individual crop diagnoses could potentially become a distributed sensing layer for detecting geographic changes in agricultural disease patterns.
That could help identify emerging problems earlier and give extension services, agricultural organizations, and researchers a richer picture of what is happening across districts.
The ultimate goal is not to replace agricultural experts.
It is to make the first layer of useful agricultural intelligence available where the problem actually begins: at the plant, in the field, in the farmer's hands.
AgriDocAI starts with one photograph.
The long-term vision is a system that helps protect crops, improve agricultural decision-making, and generate better environmental intelligence at scale.
Log in or sign up for Devpost to join the conversation.