TerraTwin: The Autonomous AI Agronomist

Protecting global food security by turning smartphones into NASA-calibrated precision agriculture labs. We combine computer vision with satellite data to detect disease and predict crop risks.

Comment

About the project

Inspiration

In India alone, smallholder farmers lose nearly 40% of their crops annually to pests and diseases that could have been prevented with early detection. While industrial farms use million-dollar sensor networks and IoT arrays, 500 million small farmers rely on intuition and luck.

We asked ourselves: Can we compress the power of a million-dollar agronomy lab into a $50 smartphone?

We didn't want to build just another "plant identification app." We wanted to build a Digital Twin for the micro-farm—an intelligent agent that understands not just what the plant looks like, but the invisible environmental factors (soil moisture, humidity, UV index) that determine its survival.

What it does

TerraTwin is a multi-modal AI system that acts as a preventative doctor for crops.

  1. Visual Diagnosis (The Eyes): The user snaps a photo of a crop. Our fine-tuned Computer Vision model analyzes the leaf patterns to identify early signs of pathogens (e.g., Early Blight, Rust, Mold).
  2. Environmental Correlation (The Context): The app pulls real-time geospatial data from the NASA POWER API based on the user's exact lat/long. It analyzes critical vectors:
    • $$R_{spread} = \text{Pathogen} \times (\text{Humidity} + \text{Wind Speed})$$
  3. Predictive Forecasting (The Brain): Instead of just telling the farmer "Your plant is sick," TerraTwin predicts: "Because humidity in your region will rise by 12% tomorrow, this fungus will spread rapidly."
  4. Actionable Prescriptions: Our LLM Agent, grounded in agricultural university papers, generates a treatment plan. It prioritizes organic/accessible remedies (like Neem oil solutions) over expensive chemicals.

How we built it

We architected TerraTwin as a modern, scalable web application with a Python-heavy AI core.

  • Frontend: Built with Next.js 14 and Tailwind CSS. We used Framer Motion for a "Bio-Futurist" UI that feels like a piece of high-tech machinery, and Mapbox GL to visualize the disease hotspots on a 3D globe.
  • Computer Vision: We utilized PyTorch and transfer learning on the MobileNetV3 architecture, trained on the PlantVillage dataset (50,000+ images of healthy/diseased crops).
  • Satellite Intelligence: We integrated the NASA POWER API to fetch metrological data (precipitation, soil moisture, radiance) in real-time.
  • The Reasoning Engine: We used FastAPI to orchestrate the logic. An OpenAI GPT-4o agent acts as the "Agronomist," synthesizing the visual confidence score with the satellite weather data to write the final report.

Challenges we ran into

  • The "Black Box" Problem: Initial AI models would confidently diagnose a healthy leaf as diseased. We implemented a "Confidence Threshold" ($$C > 0.85$$) and a fallback loop that asks the user for a second angle if the AI is unsure.
  • Data Fusion: Correlating a micro-image (a single leaf) with macro-data (satellite weather grids) was difficult. We had to normalize the NASA data to be relevant for a single farm's micro-climate.
  • Latency: Running vision inference and external API calls simultaneously was slow. We moved the API calls to run asynchronously (asyncio) while the image was uploading, cutting wait times by 60%.

Accomplishments that we're proud of

  • The "Science Fiction" UX: We managed to make the app look and feel "Legendary." The transition from the camera scan to the satellite map zoom creates a visceral connection between the ground and the sky.
  • Real-World Grounding: Unlike generic chatbots, our agent references real weather constraints. It won't recommend "drying the leaves in the sun" if the NASA API says it's raining today.
  • Accuracy: Achieving a 94% validation accuracy on the core 10 crop disease classes.

What we learned

We learned that Context is King. A leaf spot looks different in a drought vs. a flood. By fusing Computer Vision with Environmental Data, we realized that AI can stop being just a "classifier" and start being a "forecaster." We also gained deep appreciation for the complexity of the food supply chain and the resilience of farmers.

What's next for TerraTwin

  • Offline-First Architecture: Using TensorFlow Lite to run the vision model entirely on-device for farmers with no internet.
  • Drone Integration: Allowing users to upload drone footage to map an entire acre at once.
  • Community Alerts: If 5 farmers in a 1km radius detect "Fall Armyworm," TerraTwin will automatically alert all neighbors to take preventative action.

Built With

Share this project:

Updates