AgriDocAI

Inspiration

I grew up around smallholder farmers in Uganda and repeatedly saw the same bottleneck: a farmer could notice that a crop was sick, but access to agricultural expertise was often far away or delayed. By the time a problem was correctly identified, it could have spread across much of a field.

At the same time, many farmers already carry the most powerful diagnostic tool they need in their pockets: a smartphone camera.

That led to a simple question:

What if a farmer could photograph a sick leaf and receive an evidence-grounded first layer of agricultural expertise immediately?

AgriDocAI was built to explore that idea.

What it does

AgriDocAI turns a smartphone photograph into an actionable agricultural assessment in seconds.

A farmer photographs a cassava, maize, banana, or coffee leaf. The system analyzes the image, identifies likely disease symptoms, provides a diagnosis, suggests treatment, incorporates local weather conditions into its recommendations, and explains the underlying biology in plain language.

The system is designed to be more than a simple AI chatbot:

Visual symptom grounding: numbered pins identify the regions of the farmer's own image that contributed to the diagnosis.
Differential reasoning: the system shows alternative conditions considered rather than presenting only a single unexplained answer.
Uncertainty-aware diagnosis: uncertain cases are automatically escalated from Gemini Flash to Gemini Pro rather than forcing a low-confidence answer.
Confidence calibration: the platform is designed to compare predicted confidence with real farmer- or expert-confirmed outcomes over time.
Personalized learning: diagnoses become micro-lessons that build a crop-specific learning curriculum from the farmer's own cases.
Weather-aware treatment: current district-level weather informs treatment timing and recommendations.
Expert review: cases can be exported as a structured package for agronomist validation.
Multilingual access: English, Luganda, and Swahili voice output are supported.
Outbreak intelligence: opt-in, aggregated observations can contribute to a district-level outbreak map as real usage grows.

AgriDocAI is not designed to replace agricultural experts. It is designed to make a useful first layer of expertise accessible when an expert is not immediately available.

How we built it

AgriDocAI was built as a lightweight, self-contained web application using HTML, CSS, and JavaScript, with the architecture designed around low-cost and low-infrastructure deployment.

The diagnosis pipeline is layered rather than relying on a single model call.

First, an on-device HSV colour-space analysis measures the proportion of the leaf showing healthy green tissue, yellowing, and necrosis. This provides an immediate visual signal before any AI request is made.

That information is combined with a curated reference set covering major documented diseases for the selected crops, based on Uganda-focused agricultural literature, and passed to Gemini 3.5 Flash as grounding context for multimodal reasoning.

When the initial diagnosis falls below a confidence threshold, the system automatically escalates the same case to Gemini 3.1 Pro for a second review. This confidence-triggered cascade avoids using the most expensive model on every case while giving uncertain cases additional scrutiny.

The application also uses:

Firebase / Firestore for anonymous authentication, case storage, outbreak data, and calibration infrastructure.

Open-Meteo for current district-level weather information.

Puter.js as a fallback AI access route when a direct Gemini API key is unavailable.

The result is a system that combines computer vision, multimodal AI, retrieval-grounded reasoning, uncertainty handling, weather data, and longitudinal learning within a lightweight browser application.

Challenges we ran into

One of the biggest challenges was designing around real infrastructure constraints rather than assuming unlimited compute, storage, or API access.

Our first approach relied heavily on a free AI proxy, but image-heavy usage exposed storage and reliability limitations. That led to a hybrid architecture with direct Gemini API access and an automatic fallback route.

The second challenge was uncertainty.

It is easy to display a confidence percentage. It is much harder to make that percentage meaningful. Rather than treating a model's confidence output as ground truth, we built infrastructure to eventually compare predicted confidence against real outcomes.

We also deliberately avoided fabricating data. During development, we considered making the outbreak map appear more populated using synthetic cross-user reports. We rejected that approach because a fabricated disease signal could encourage a real agricultural decision based on false information. Sample data is therefore clearly identified as sample data.

Finally, spatial symptom annotations required careful validation. Vision models do not always return reliable coordinates, so the system checks whether returned coordinates are consistent with the requested output and falls back to text-only explanations when they are not.

Accomplishments that we're proud of

We are most proud that AgriDocAI became more than a basic image-classification demo.

We built an end-to-end agricultural diagnostic system incorporating:

on-device visual preprocessing → grounded multimodal reasoning → uncertainty-triggered escalation → treatment recommendations → weather context → explainability → expert review → confidence calibration infrastructure.

We also built the system as a solo developer on essentially zero-cost infrastructure, while deliberately prioritizing honest system behaviour over features that merely make a demo look impressive.

The most important design principle became simple:

If the system does not know, it should not pretend that it does.

What we learned

The biggest lesson was that building an AI product is not the same as calling an AI model.

Our initial idea was to use several models simultaneously and let them vote on every diagnosis. We replaced that with a confidence-triggered cascade because it was faster, cheaper, and better aligned with the actual decision-making problem.

We also learned that calibration matters more than impressive-looking confidence numbers. A system that says "60%" and is correct roughly 60% of the time is more useful than one that confidently says "95%" while being correct substantially less often.

Most importantly, we learned that responsible engineering sometimes means deliberately making a demo look less impressive. We chose transparent sample data, explicit uncertainty, and validation pathways over fabricated evidence or unsupported claims.

What's next for AgriDocAI

The next major step is formal validation.

We plan to evaluate AgriDocAI against labeled crop-disease datasets and, more importantly, obtain expert-reviewed cases from agricultural professionals to measure diagnostic accuracy, calibration, and failure modes.

Beyond validation, the roadmap includes expanding the disease reference set, supporting additional crops and districts, and developing SMS/USSD access for farmers with unreliable mobile data.

As real usage grows, the same infrastructure could support aggregated district-level disease surveillance while preserving farmer privacy.

The long-term goal is not to replace agricultural extension services.

It is to create a scalable first layer of agricultural intelligence that can help a farmer identify a problem, understand it, and decide what to do next—before the problem becomes a larger one.