AgroMind

Simulation Dashboard
Computer Vision
Configuration
Recommendation Modal

🌱 Inspiration

Greenhouse disease can spread quickly, and growers often do not know there is a problem until symptoms are already visible across multiple plants. By that point, treatment becomes less targeted, more expensive, and more wasteful.

We wanted to build a system that makes greenhouse monitoring more proactive. Instead of waiting for disease to spread, AgroMind combines environmental conditions, autonomous scouting robots, computer vision, and crop-health modeling to help detect disease earlier and recommend where attention is needed.

AgroMind was inspired by one question: what if a greenhouse could maintain a live understanding of crop health and use scouting robots to inspect the highest-risk plants first?

🍅 What it does

AgroMind is an AI greenhouse intelligence platform for tomato disease monitoring.

Our project combines a 3D greenhouse simulation with a real tomato leaf disease classifier. Users can generate a greenhouse, configure environmental conditions, run autonomous robot scouting, inspect tomato plants, classify leaf images, update a live belief map, and view recommendations for what the system should do next.

The system includes:

Configurable greenhouse conditions like humidity, temperature, light, and soil moisture
A 3D greenhouse simulation with tomato plants and scouting robots
Hidden disease pressure generated from environmental risk and spatial hotspots
A real tomato disease classifier trained on PlantVillage tomato leaf images
FastAPI endpoints for image prediction
A crop-health belief map that updates after inspections
Agent logs, scouting metrics, sensor panels, and intervention recommendations

Instead of treating the whole greenhouse the same way, AgroMind helps identify where disease is likely forming and where targeted inspection or treatment should happen first.

🛠️ How we built it

We built AgroMind as two connected parts: a greenhouse intelligence frontend and a computer vision backend.

The frontend is built with Next.js, React, TypeScript, Tailwind CSS, Three.js, and React Three Fiber. It renders a 3D greenhouse with plant rows, tomato plants, risk overlays, and rover-style scouting robots. The dashboard shows the real simulated greenhouse next to AgroMind’s belief map, so users can see how the system’s understanding changes as inspections happen.

The greenhouse simulation uses environmental inputs to generate hidden disease pressure. Humidity, soil moisture, low light, and temperature all contribute to risk. A simplified version of the environmental risk model is:

[ EnvironmentRisk = 0.35H + 0.25M + 0.20L_{low} + 0.20T ]

where (H) is humidity risk, (M) is soil moisture risk, (L_{low}) is low-light stress, and (T) is temperature-related disease pressure.

Each plant receives a hidden disease state and an image associated with one of four tomato classes:

Healthy
Early blight
Late blight
Leaf mold

For the computer vision backend, we trained a real tomato leaf classifier using TensorFlow/Keras and MobileNetV2. The model trains on four PlantVillage tomato classes and outputs a prediction, confidence score, and severity value. The backend exposes predictions through a FastAPI service so robot-captured images can be classified through an API.

Example model response:

{
  "prediction": "early_blight",
  "confidence": 0.8732,
  "severity": 0.7
}

The scouting agent chooses which plant to inspect using a scoring function based on nearby disease signals, distance, and greenhouse position. A simplified version is:

[ TargetScore = 0.5N + 0.3C + 0.2D ]

where (N) is nearby disease belief signal, (C) is center priority, and (D) is distance efficiency.

When a robot inspects a plant, the system updates the plant’s prediction, confidence, belief risk, metrics, logs, and next-step recommendation.

🌧️ Challenges we ran into

One of the biggest challenges was connecting multiple systems into one coherent product. AgroMind is not just a classifier and not just a simulation. We had to connect environmental risk modeling, robot scouting, image classification, belief-map updates, and dashboard recommendations into one workflow.

Another challenge was scope. The full vision includes real greenhouse sensors, real robots, trained computer vision models, digital twins, and automated greenhouse actuators. For the hackathon, we focused on building the strongest demoable version: a 3D greenhouse simulation connected to a real tomato disease classification backend.

Training the CV model also required careful setup. We built a standalone TensorFlow/Keras pipeline, prepared deterministic train/validation/test splits, trained MobileNetV2, generated model artifacts, and exposed inference through FastAPI. We also had to be careful not to overclaim model performance, because PlantVillage images have clean backgrounds and do not fully represent messy real greenhouse conditions.

The 3D simulation was also challenging. Early versions looked too flat and grid-like, so we moved to a Three.js-based greenhouse with plant rows, robot movement, risk overlays, zoom controls, and a clearer split between physical ground truth and the system’s belief map.

🏆 Accomplishments that we're proud of

We are proud that AgroMind demonstrates a full AI workflow instead of only showing one isolated model.

The project includes a real trained tomato disease classifier, a FastAPI inference service, a 3D greenhouse simulation, autonomous robot scouting, belief-map updates, metrics, and recommendations. Together, these pieces show how computer vision can become part of a larger greenhouse decision-support system.

We are also proud that the simulation makes the AI loop easy to understand visually. Users can configure the greenhouse, run scouting, inspect plants, see classification results, watch the belief map update, and compare the system’s understanding against the hidden disease scenario.

Most importantly, AgroMind is built around a real agricultural problem: detecting crop disease earlier and reducing unnecessary whole-greenhouse treatment.

🧠 What we learned

We learned that the value of AI in agriculture is not only the model prediction. A classifier can identify disease in one image, but the real product value comes from connecting that prediction to plant location, environmental conditions, scouting behavior, uncertainty, and recommended action.

We also learned how important simulation is for robotics and agriculture. Even before deploying real robot hardware, a simulation lets us test scouting strategies, disease-risk modeling, robot movement, and operator workflows.

On the computer vision side, we learned how to build a practical model pipeline: dataset preparation, deterministic splits, transfer learning with MobileNetV2, model evaluation, artifact generation, and API deployment.

We also learned to be honest about model limits. A model trained on PlantVillage can perform well on clean test images, but real greenhouse images may include occlusion, lighting changes, motion blur, overlapping leaves, and background noise. Future versions need greenhouse-specific validation before making real-world accuracy claims.

🚜 What's next for AgroMind

Next, AgroMind will expand from a tomato-focused demo into a crop-agnostic greenhouse intelligence platform. The system is designed so any crop can be supported as long as there is a compatible disease dataset and trained model.

The next major step is connecting the full greenhouse response loop. Instead of only detecting disease and showing recommendations, AgroMind should coordinate simulated or real greenhouse actuators such as humidifiers, ventilation systems, irrigation, medicine sprayers, and targeted treatment tools. This would move the platform from monitoring toward closed-loop intervention.

Future versions can also add a model registry for multiple crops, real sensor adapters, stronger robot hardware integration, and more advanced predictive planning. Long term, AgroMind could become a full greenhouse operating system that combines sensors, autonomous scouting, computer vision, crop-health modeling, and automated intervention to detect disease earlier, reduce unnecessary pesticide use, and help growers manage outbreaks before they spread.