Field Agent

Dashboard Home
Health Check for Crop Fields
Agentic AI for questions about Farmer's Field
Landing Page
Health Check for Crop Fields Continued
Health Check for Crop Fields Continued
Health Check for Crop Fields Continued
Eleven Labs Voice Demo
SMS Demo
Health Check for Crop Fields

Inspiration

Farm income in the US depends on daily, high-stakes decisions — when to irrigate, when to fertilize, when to respond to crop stress. A single missed irrigation call on a cotton field in Arizona can cost hundreds of dollars per acre. Yet according to the 2022 USDA Census of Agriculture, the US now has fewer than 2 million farms for the first time since before the Civil War, and 86% of them are small family operations with average income that's actually negative (−$2,976/year per USDA ERS data). These are the farmers who need decision support the most — and they're the ones who have the least access to it.

The data to make better decisions exists. USGS Landsat satellites capture crop health (NDVI) at 30-meter resolution every 8 days — for free. NOAA and OpenWeatherMap provide real-time weather. The USDA Web Soil Survey covers 95%+ of US counties with detailed soil profiles. But today, that data lives in separate government portals that no working farmer has time to cross-reference.

Meanwhile, the human advisory system is collapsing. USDA Cooperative Extension county agents declined 30% between 1980 and 2010 — from 11,441 to 7,974 FTEs — leaving roughly 1 agent for every 236 farms (Choices Magazine / USDA ERS). In Arizona specifically, the University of Arizona Cooperative Extension faces a 3% budget cut for FY2026, and critical positions like a cotton weed science specialist remain unfilled.

We built Field Agent because we saw a clear gap: the data is free, the farmers are struggling, and no existing tool closes the loop from raw data to "here's exactly what you should do and what it'll cost."

What It Does

Field Agent is a Gemini-powered autonomous AI agent that a farmer can text or call. The agent executes a full sense → reason → act → notify loop:

Sense — On receiving a message, the agent autonomously calls tools to gather real-time weather data (OpenWeatherMap), satellite-derived crop health via NDVI (sourced from real USGS/Landsat imagery for an actual cotton field near Casa Grande, AZ), and USDA soil profile data.
Reason — Gemini 2.5 Flash synthesizes all inputs using function calling, considers crop type, growth stage, weather forecast, and soil properties to determine the optimal action.
Act — The agent generates a specific recommendation with estimated dollar cost, urgency level, and quantified risk of inaction.
Notify — The recommendation is delivered via SMS/WhatsApp (Twilio) or spoken directly to the farmer during a live phone call powered by ElevenLabs Conversational AI.

A farmer's entire interaction is one text message in, one action plan out. Example: "How's my field looking?" → "Your cotton is showing early drought stress. Irrigate within 24 hours. Estimated water cost: $45. Delaying 3+ days risks 12% yield loss."

The Problem: Real Numbers, Real Stakes

Case Study: Pinal County, Arizona — Top Yields, Bottom Water Security

Our demo targets a real cotton field near Casa Grande in Pinal County, AZ — Arizona's #1 cotton-producing county. According to USDA NASS data, Pinal County produced 176,500 bales on 56,800 acres in 2022, yielding 1,511 lbs/acre — roughly 60% above the national average. Cotton contributes $322 million to Arizona's economy (University of Arizona Cooperative Extension).

But this productivity sits on top of an existential water crisis. Arizona faces a Tier 1 Colorado River shortage with a 512,000 acre-foot reduction — roughly 30% of the Central Arizona Project's normal supply. Cotton requires approximately 4.2 acre-feet per acre, and CAP agricultural water now costs $95/acre-foot (2025 rate schedule), making water alone ~$399/acre — over half of total production costs (~$736/acre). The Arizona Department of Water Resources has stated that Pinal groundwater demand exceeds sustainable supply by 8 million acre-feet over 100 years.

Every irrigation decision matters. The University of Arizona's $63M Water Irrigation Efficiency Program demonstrated that switching from flood to drip irrigation can save 36,418 acre-feet annually. Smart irrigation scheduling alone can save 30–50% of water use while improving yields by 20–30% (ScienceDirect meta-analysis, 2025). An AI agent that tells a farmer exactly when and how much to irrigate isn't a convenience — it's an economic lifeline.

The Precision Agriculture Gap

The USDA ERS's 2023 ARMS survey shows the adoption divide clearly: large-scale farms use yield monitors at 68% and variable-rate technology at 45%, while small farms sit at 13% and 5% respectively. A 2024 GAO technology assessment (GAO-24-105962) confirmed three structural barriers: prohibitive upfront costs, data ownership concerns, and lack of interoperability. Only 27% of all US farms used any precision agriculture practice in 2022–2023.

The cost of this gap is measurable. The American Farm Bureau Federation estimates weather-related crop losses at $21+ billion annually. USDA crop insurance indemnities hit $19.1 billion in 2022 alone, with cumulative payouts exceeding $161 billion from 2001–2022 (USDA RMA / EWG). A peer-reviewed study in Agricultural and Forest Meteorology found that management-driven yield gaps account for roughly 22% of potential rainfed soybean yield in the North-Central US — driven by sowing date, tillage, and pest response timing. These are precisely the decision points an AI advisor can optimize.

What Competitors Lack — And What We Do Differently

Platform	What it does	What it doesn't do
Climate FieldView (Bayer)	220M+ subscribed acres, yield maps, satellite imagery, variable-rate seed scripts	Doesn't tell the farmer what to do today. Farmer forum feedback: "a bunch of pretty pictures don't do anything for making money."
FarmLogs (acq. by Bushel, 2021)	Operational & financial dashboards across 110M acres	Stopped at visualization. Left interpretation and action decisions entirely to the farmer.
Granular (Corteva/DowDuPont)	Farm management analytics	Corteva shut down Granular Agronomy in Aug 2022 and sold the unit — explicitly moving away from SaaS. Dashboard-only products couldn't sustain commercial viability.
OSU ExtensionBot (2024)	AI chatbot over 400K+ extension publications	Answers general questions from publications. No real-time field data, no personalized prescriptions.
Farmer.Chat (Digital Green/Microsoft)	RAG-based chatbot, 15K+ farmers, 300K+ queries	Focused on developing countries. No satellite/NDVI integration, no cost estimation, no US-specific data.
Field Agent (OUR PROJECT)	Gemini agent with function calling, live weather + real NDVI + USDA soil → specific, costed action plans via SMS or voice call	This is what we do. One message in, one decision out. Works on a $20 phone.

The key differentiator: every prior product stops at the dashboard. Field Agent closes the loop. It doesn't show data — it makes decisions and tells the farmer exactly what to do and what it costs. The reasoning trace dashboard proves the agent is thinking, not just pattern-matching.

How We Built It

Agent Brain: Gemini 2.5 with function calling — the agent autonomously decides which tools to call based on the farmer's natural language input
Three Tool Functions: get_weather (OpenWeatherMap API — live), get_crop_health (NDVI data sourced from real USGS/Landsat imagery for Casa Grande), get_soil_profile (USDA Web Soil Survey data)
Backend: Django 5 + Django REST Framework for API layer and agent orchestration
Frontend: Next.js 14 (App Router) — reasoning trace dashboard showing which tools the agent called, what data came back, and how it made its decision
Messaging: Twilio SMS/WhatsApp for the farmer-facing interface
Voice: ElevenLabs Conversational AI — farmer can call a phone number and have a live spoken conversation with the same agent, using the same tools. Hands-free, literacy-independent, works while driving a tractor.
Hosting: Google Cloud Platform and Heroku

Data Sources (All Free, All Real)

Dataset	Source	What it provides
Crop Health (NDVI)	USGS Landsat 8/9	Vegetation stress index (0–1), 30m resolution, 8-day revisit
Weather	OpenWeatherMap API	Real-time temp, humidity, wind, 7-day precipitation forecast
Soil Profiles	USDA NRCS Web Soil Survey (SSURGO)	Soil type, pH, drainage class, water-holding capacity
Crop Statistics	USDA NASS Quick Stats	County-level yield, acreage, production data

Challenges We Faced

Satellite data latency vs. hackathon timeline: Live Landsat integration via Google Earth Engine requires OAuth service account setup and tile processing that exceeds a 48-hour window. We solved this by pre-fetching real USGS NDVI readings for an actual Casa Grande cotton field and serving them as static data — honest about the limitation, with live satellite integration on the roadmap via GEE.
Making Gemini's function-calling loop reliable: The agent sometimes wanted to call tools in suboptimal order or skip the synthesis step. We iterated heavily on the system prompt to ensure it follows the sense → reason → act chain consistently.
Voice agent tool integration: Getting ElevenLabs Conversational AI to call the same backend tools in real-time during a live phone call required careful endpoint design — the voice agent needs low-latency responses while the Gemini agent needs time to reason.
Cost estimation accuracy: Rule-based cost calculations (water cost × volume × field size) are useful for demos but don't capture the full complexity of real farm economics. We scoped this as a known limitation with plans for actual market price integration.

What We Learned

The extension gap is worse than we expected. When we dug into the numbers — 1 agent per 236 farms, 30% decline in staffing, state budget cuts actively eliminating positions — it became clear this isn't a future problem. Farmers are making thousand-dollar decisions with zero expert input right now.
The data is there; the interface isn't. Every dataset Field Agent needs is freely available via US government APIs. The bottleneck was never data availability — it's that no one has packaged it into a format a farmer can use from a $20 phone while standing in a field.
AI-driven irrigation scheduling works. Peer-reviewed evidence shows 10–25% yield improvements and 20–50% water savings in arid/semi-arid conditions comparable to Arizona. The research validates the approach; the challenge is access and delivery.
Voice-first matters more than we thought. A farmer on a tractor can't read a text. A farmer with low literacy can't parse a dashboard. ElevenLabs Conversational AI turned our SMS agent into something that genuinely solves an access problem — and that realization reshaped our entire product thesis.