VirgoEye

• 

  VirgoEye’s main inspection dashboard

• 

  The core four-step inspection workflow

Inspiration Farmers, agronomists, and field workers often have to make high-stakes crop health decisions based on a single, imperfect smartphone photograph. Existing tools often act as opaque black boxes that overpromise definitive diagnoses from limited visual data. VirgoEye was inspired by the need for a transparent, practical field-check assistant that maps visible evidence, highlights uncertainty, and supports human judgment rather than replacing it.

What it does VirgoEye turns any crop photograph into structured visible observations, approximate image region mappings, and practical next inspection steps. Key features include:

Accessible Input: Allows uploading crop photos or choosing from sourced PlantVillage examples and blurred test cases.

Live Streaming Analysis: Streams Anthropic-backed vision request progress directly into the browser.

Inspectable Evidence: Lets users select detected regions and compare observations against the image using approximate bounding boxes.

Deterministic Safeguards: Prevents unverified or rejected findings from being presented as an all-clear, replacing empty action text with disclosed general guidance.

Portable Reports: Generates standalone HTML reports containing the photo, evidence references, and limitations for browser viewing or PDF printing.

How we built it Tech Stack: Built using a FastAPI Python backend, Next.js 15, React 19, TypeScript, and Tailwind CSS for the frontend interface.

AI & Orchestration: Utilized Pydantic schemas for strict structured outputs, server-sent events for real-time progress, and Anthropic vision requests split across multi-stage inspection steps (with an optional automated critic pass).

Robust Testing: Implemented automated backend tests, report export checks, type checking, and simulation-mode smoke tests to ensure CI reliability without requiring paid API keys.

Challenges we ran into Managing Model Limitations: Balancing structured AI outputs with strict review safeguards so that model uncertainty, uncalibrated confidence scores, and approximate region boxes are clearly communicated to the user.

Multi-Stage Orchestration: Coordinating sequential vision requests (mapping, observations, optional critic, and synthesis) while maintaining smooth streaming progress and caching state.

Controlled vs. Field Data: Accounting for the fact that controlled leaf datasets (like PlantVillage) do not represent messy field deployment conditions, requiring explicit disclaimers in the UI and reports.

Accomplishments that we're proud of Successfully engineering a deterministic review policy that intercepts poor or template-only model outputs and replaces them with safe general guidance.

Building a fully portable, standalone HTML report generator that packages photos, evidence, and limitations into a single offline-viewable file.

Creating a robust simulation mode (VIRGO_DEMO_MODE=true) that exercises the entire UI with synthetic, image-aware output without requiring live provider keys or incurring charges.

What we learned Single-image photography is inherently limited; true agricultural assessment requires acknowledging uncertainty rather than masking it behind confident AI guesses.

Implementing strict Pydantic validation and server-sent events significantly improves the reliability and responsiveness of multi-agent vision pipelines.

What's next for VirgoEye Expanding beyond controlled leaf datasets to broader field-testing conditions with agronomist review integration.

Transitioning experimental local inference paths (like Ollama or vLLM adapters) into validated capabilities.

Securing proper public hosting, persistent budgeting, operational monitoring, and HTTPS deployment verification.

Built With

• and
• and-multi-stage-inspection/critic-pipelines.-testing-&-infrastructure:-pytest-for-automated-backend-testing
• and-pydantic-schemas-for-structured-outputs.-frontend:-next.js-15
• and-tailwind-css.-ai-&-architecture:-anthropic-backed-vision-requests
• backend:-fastapi
• experimental
• npm-test-suites-for-type-checking-and-report-exports
• ollama
• python-3.12+
• react-19
• server-sent-events-for-streaming-progress
• typescript
• vllm
• with-optional-docker