NeuroMed: Agentic Clinical AI

Autonomous agent that cross-references reports with Fivetran EHR.

Comment

🩺 The Inspiration

Modern healthcare generates massive volumes of unstructured clinical text. We realized that current medical AI solutions are essentially just glorified chatbots. They lack tool-use, deterministic reasoning, and most importantly, historical context. Missing a critical contradiction between a newly uploaded pathology report and a deeply buried historical Electronic Health Record (EHR) can be fatal. We wanted to build an Agent that doesn't just chat, but actively reasons and executes clinical safety protocols.

⚙️ What it does

NeuroMed is an autonomous Agentic Reasoning Engine.

  1. Contextual Analysis: It ingests complex, unstructured medical PDFs and immediately triggers a Fivetran MCP connector to fetch historical patient records.
  2. Cross-Referencing: It cross-references the live report with the Fivetran EHR data to detect critical anomalies (e.g., contraindicated medications).
  3. Automated Escalation: If a critical risk is flagged, the agent bypasses passive chat and directly triggers a GitLab webhook to create a priority escalation ticket for the clinical board.
  4. Clinical Copilot: Provides a live conversational interface powered by Gemini 1.5 Flash for doctors to query the entire agentic context.

🏗️ How we built it

We utilized a multi-agent deterministic orchestration architecture:

  • The Brain: Powered by Gemini 1.5 Pro via Google Cloud Vertex AI for deep clinical entity extraction and reasoning.
  • Tool Integration: Integrated Model Context Protocol (MCP) servers specifically focusing on the Fivetran track to sync patient history, and GitLab for incident management.
  • Self-Healing & Observability: We didn't want a "black box" AI. We integrated Arize Phoenix for full LLM tracing. We built a secondary 'Critic Agent' that evaluates the primary output against an 85% confidence threshold, automatically triggering a time.sleep() retry and re-prompting loop if the clinical logic fails.
  • The Stack: The backend is engineered in Python (FastAPI) to handle asynchronous tool calls, wired seamlessly to a high-fidelity React frontend showcasing live dynamic telemetry.

🚧 Challenges we ran into

Building a deterministic agent is hard. Our biggest hurdle was handling API rate limits and model restrictions on the chat endpoint which resulted in 500 Server Errors during live testing. We overcame this by engineering a bulletproof 3-tier fallback execution loop in FastAPI, ensuring the system dynamically degrades gracefully to stable models without breaking the frontend UI. We also spent significant time successfully wiring the frontend observability switches directly to the backend logic to create a truly dynamic demo experience.

🚀 What's next for NeuroMed

We plan to expand our MCP toolset to include live IoT vitals monitoring and transition our self-healing loop from a mock threshold to a fully fine-tuned medical evaluation model.

Built With

Share this project:

Updates

Submission history