NutriGuard AI

AI health copilot: OCR-extracts profiles, maps destinations, prevents drug-food risks, provides live market prices via Fivetran, & generates multilingual medical alert cards.

Comment

💡 Inspiration

Traveling with chronic conditions or severe allergies shouldn't be a gamble. We were inspired by the gap between global travel and localized, real-time health intelligence. Existing apps provide static advice, but they fail to account for the culinary chemistry of a destination (e.g., Japanese ingredients impacting blood thinners) or the economic reality of food access. We wanted to build a "Copilot" that doesn't just inform, but actively protects travelers.

🚀 What it does

NutriGuard AI is an autonomous travel health copilot that bridges the gap between medical records and destination safety.

  • Intelligent OCR: Processes complex medical documents (PDFs/Images) into structured health profiles.
  • Agentic Reasoning: Orchestrates four specialized AI agents (Health, Nutrition, Travel, Safety) to cross-reference dietary needs with local cuisine and global travel advisories.
  • Fivetran Data Superpowers: Uses Fivetran to stream live agricultural commodity pricing into Google BigQuery, allowing the agent to provide budget-aware meal safety advice.
  • Emergency Readiness: Generates real-time, multilingual Medical Alert Cards and finds verified local healthcare facilities via Google Maps.

🛠️ How we built it

  • Orchestration: Google Cloud Agent Builder to orchestrate multi-agent workflows.
  • Intelligence Layer: Google Gemini 2.5 Flash for reasoning, OCR, and intent classification.
  • Data Pipeline (Fivetran Track): We leveraged Fivetran to ingest live Agmarknet market data and WHO advisories into BigQuery. Our agents interact with this warehouse via MCP and REST APIs to provide context-aware insights.
  • Model Fallback: Implemented a resilient cascade architecture (Gemini ➔ SambaNova Llama 3.3-70B ➔ Regex) to ensure 99.9% uptime despite API rate limits.
  • Frontend: A responsive UI using FastAPI and a premium, agent-optimized dashboard. ## System Architecture NutriGuard Architecture

🚧 Challenges we ran into

  • API Rate Limits: Running four agents simultaneously often hit Gemini's free-tier limits. We solved this by building a multi-provider fallback system that switches to SambaNova's Llama models instantly.
  • Data Normalization: Mapping disparate APIs (Agmarknet, DrugBank, USDA) was complex. Fivetran allowed us to normalize this data into clean BigQuery tables, simplifying our SQL queries significantly.
  • AI Hallucinations: When agents didn't have data, they would sometimes guess. We implemented strict JSON-schema enforcement and database-grounding protocols to ensure the AI only speaks from verified tables.

🏆 Accomplishments that we're proud of

  • Resilience: Building an app that handles API exhaustion seamlessly without crashing.
  • Real-time Pipeline: Successfully creating an end-to-end data pipeline where live agricultural data influences AI-based nutritional recommendations.
  • Clinical Accuracy: Achieving high-fidelity OCR extraction from complex, non-standardized medical documents.

🧠 What we learned

  • Agents > Chatbots: We learned that the real value isn't the chat—it's the agent's ability to "plan" (e.g., Check Fivetran sync status ➔ Query BigQuery ➔ Translate Output).
  • Data Freshness: Real-world data (prices, hospital locations) requires a robust ingestion strategy. Fivetran transformed our prototype into a functional data-aware product.

🔮 What's next for NutriGuard AI

  • Wearable Integration: Syncing real-time glucose and heart-rate data for predictive health alerts.
  • Global Fivetran Expansion: Integrating additional Fivetran connectors to include regional pharmacy inventory and insurance coverage databases.
  • Offline Mode: Deploying a localized, small-language-model (SLM) version of the health agent for travelers in areas with zero connectivity.
Share this project:

Updates