Inspiration
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for FieldSense Commerce AIInspiration
Traditional crop diagnosis apps suffer from a fundamental, costly flaw: the hasty single-photo guess. A grower uploads a photo of a yellowing leaf, and a naive computer vision model immediately returns a 98% confidence prediction of a fungal disease, advising the grower to buy chemical fungicides. In reality, that foliar chlorosis is far more likely caused by root zone anoxia from recent heavy rainfall, nitrogen translocation, or pH-induced cation lockout. Spraying chemicals when the problem is waterlogging damages soil biology, wastes capital, and fails to solve the root issue.
We built FieldSense Commerce AI to reverse this paradigm. Instead of guessing from an isolated photo, FieldSense behaves like an experienced field agronomist: formulating multiple competing scientific hypotheses, prescribing non-destructive physical discriminating tests in the dirt, and connecting verified findings directly to server-authoritative remediation supplies powered by the PayPal REST API v2.
What it does
FieldSense Commerce AI is an evidence-based agricultural intelligence and commerce platform that turns raw field observations into scientifically grounded diagnoses and secure supply orders:
Multimodal Field Investigation: Growers record crop symptoms via photos, qualitative field notes, and ecological context (soil moisture, recent weather patterns, anatomical distribution). Multi-Agent Scientific Reasoning: The AI coordinates specialist agents across Plant Pathology, Soil Agronomy, and Micro-Climate Ecology to propose competing hypotheses with concrete supporting vs. contradicting evidence. Prescribed Physical Discriminating Tests: Directs the grower outdoors to execute concrete physical tests (e.g., non-destructive soil ribbon tests, root-core olfactory inspection, lower cambium ring scrapes) to definitively separate competing causes. Authoritative Catalogue Matchmaking: Diagnostic hypotheses are mapped to an authentic catalogue of diagnostic reagents, loupes, and biological bio-fertilizers. Fail-Closed PayPal v2 Checkout: A slide-out remediation drawer computes server-authoritative quotes and enables in-context purchasing through PayPal Smart Payment Buttons. Cryptographic Order Reconciliation: Features strict server-side order binding, currency/amount verification, tamper-proof capture handling, and cryptographic webhook signature verification (verify-webhook-signature) that updates order history with real PayPal transaction receipts. How we built it
FieldSense Commerce AI was engineered with a modern, decoupled, and secure full-stack architecture:
Frontend Client (React 19, TypeScript, Vite): Built with React 19 and @paypal/react-paypal-js for a fluid field observer experience, interactive mission checklists, and an in-context supply drawer. Designed with synchronous mutex reference locking (useRef) to prevent duplicate order creation, lifecycle race conditions, or unvalidated client returns. Backend API & Commerce Engine (FastAPI, Python 3.11): Server-Authoritative Quoting: Calculates item pricing, shipping tiers, and regional taxes strictly against backend catalogue data (catalogue.json), rejecting client-side price manipulation. PayPal Client Adapter (backend/paypal_client.py): Directly interfaces with PayPal REST API v2 (/v1/oauth2/token, /v2/checkout/orders, and /v2/checkout/orders/{id}/capture). Operates completely fail-closed without synthetic mocks in runtime. Order-Binding & Capture Validation: Binds local database orders to specific PayPal order IDs, verifies status COMPLETED, checks capture amounts to the exact penny, and enforces unit custom references. Cryptographic Webhooks: Validates all PayPal security headers (PAYPAL-AUTH-ALGO, PAYPAL-CERT-URL, etc.) via PayPal's /v1/notifications/verify-webhook-signature endpoint, with SQLite event uniqueness guarantees to eliminate duplicate execution. AI & Agentic Orchestration (Ollama & LangGraph): Primary Tier (Ollama llama3.1:8b): Delivers multimodal LLM inference with structured JSON formatting enforcing strict Pydantic schemas. Secondary Tier (LangGraph State Graph): If Ollama is offline, a compiled 6-node state graph executes deterministic agronomic heuristics, explicitly tagging output with a transparent langgraph_offline_heuristics banner. Persistence & Containerization: Powered by SQLModel/SQLite with WAL journaling and packaged via Docker and Docker Compose for instant reproducible deployment. Challenges we ran into PayPal SDK Lifecycle & Asynchronous React State Races: During fast checkout cycles, React async state updates between createOrder and onApprove could create orphaned backend orders or duplicate PayPal calls. We resolved this by implementing synchronous reference locks (pendingOrderIdRef and pendingPayPalOrderIdRef) and a UI mutex state (isProcessingPayment) that ensures a 1:1 binding between the local order and the PayPal capture. Preventing Client-Side Payment Tampering: Early architectures often rely on the frontend to pass order totals to the payment gateway. We eliminated this vulnerability: the frontend can only request a quote for SKUs and quantities. The server computes the total against the immutable backend catalogue, transmits it to PayPal, and validates the captured amount upon return. End-to-End Cryptographic Webhook Security: Handling webhook verification requires capturing raw, unparsed request byte bodies and verifying PayPal's transmission headers against their upstream verification endpoint. We built a dedicated verification pipeline with database-level event deduplication to ensure webhook replays cannot trigger double-fulfillment. Structured Multimodal LLM Consistency: Local LLMs can drift or hallucinate unstructured text. We solved this with rigorous Pydantic response schemas, regex-assisted JSON parsing, and a deterministic LangGraph fallback that guarantees the application never crashes even when local AI models are unavailable. Accomplishments that we're proud of 100% Fail-Closed, Zero-Simulation Payment Pipeline: Completely eliminated synthetic "mock payments", fake capture IDs, and fake tokens. If credentials are missing, the runtime safely refuses to process payments, displaying actionable guidance to the user. Server-Side Order Binding: The backend cryptographically and relationally validates that the PayPal order approved in the browser matches the exact local order in the database before capturing funds. Comprehensive Test Coverage: 17 backend automated pytest suites and 17 frontend Vitest test suites (34 total) passing with 100% success across order binding, tampering rejection, and webhook deduplication. Clean Production Build & Linter: 0 TypeScript compilation errors (tsc -b), 0 Vite bundler errors, and 0 Oxlint errors across all 26 source files. Agricultural Transparency: Honest AI that explicitly acknowledges unknowns, formulates competing hypotheses, labels offline modes, and encourages human extension review. What we learned Security in Agentic Commerce: Autonomous AI agents and commerce systems cannot be trusted to independently execute money movement without explicit human buyer authorization gates and server-authoritative validation. PayPal v2 API Precision: We learned the deep mechanics of PayPal v2 purchase unit custom IDs, reference binding, and idempotent capture handling. Designing payments to fail closed makes the difference between a toy prototype and production-grade commerce software. Multimodal Agronomy: Combining computer vision with structured ecological context (soil moisture, rainfall history, symptom distribution) yields vastly superior diagnostic accuracy compared to processing images in isolation. What's next for FieldSense Commerce AI PayPal Subscriptions & Soil Monitoring Kits: Introduce recurring seasonal delivery of consumable soil testing reagents, microbial inoculants, and pH test strips. Multi-Tenant Agronomist Portals: Implement role-based farmer/agronomist accounts with PayPal Payouts, allowing cooperative agronomists to review field observations and earn consultation fees. Edge Drone & Camera Integration: Expand the multimodal pipeline to ingest aerial multispectral drone imagery (NDVI/NDRE bands) and automatically generate localized supply quotes for variable-rate field applications. Offline-First PWA Synchronization: Enable progressive web app caching with local SQLite synchronization, allowing growers to record field notes completely offline in remote pastures and sync orders automatically upon reconnecting to cellular service.
