Doctor AI Agents

Patient-Centric Clinical and Disease Diagnosis with Treatment Optimization

Comment

Inspiration

The inspiration for building the Doctor AI Agent came from witnessing the complexity, delays, and inconsistencies in oncology care—particularly in cancer diagnosis, treatment planning, and follow-ups. I observed the growing need for: Precision medicine in clinical decision-making A holistic, patient-centric platform that consolidates diverse medical data Scalable solutions to reduce the burden on doctors, while improving outcomes for patients We envisioned an intelligent agent system that goes beyond symptom checkers and integrates structured (EHR, genomics) and unstructured (doctor notes, pathology reports) data to provide real-time, explainable, and optimized treatment recommendations.

What it does

Doctor AI Agent is an intelligent, multi-agent system that helps clinicians: Collect and process patient data from EHR, genomics, imaging, and pathology reports Perform AI-assisted disease diagnosis, particularly in oncology Recommend personalized treatment plans based on patient profile, clinical guidelines, and prior outcomes Match eligible patients to clinical trials using biomarkers, disease stage, and location Provide explainable outputs for better doctor-patient communication It acts like a virtual tumor board, empowering clinicians with evidence-backed decision support.

How we built it

We developed the system on Google Cloud Platform (GCP) using a robust Agentic AI architecture. Key components include: Technologies & Tools Cloud: GCP (Vertex AI, BigQuery, Healthcare API, Cloud Storage, Cloud Run), Python LLMs: Gemini-2.0-flash-001 with RAG, (optional-Med-PaLM 2) Orchestration: Agent Development Kits ( ADK), LangGraph, Vertex AI ETL/Data Pipelines: Cloud Storage, Apache Beam, Cloud Dataflow Frontend: ADK's UI with Doctor Mode interface

Challenges we ran into

Handling sensitive patient data securely LLM hallucination and clinical accuracy Data heterogeneity (structured vs. unstructured) Orchestrating multi-agent workflows Gaining physician trust

Accomplishments that we're proud of

Successfully integrated multimodal clinical data for over 100 simulated patient cases Achieved over 85% accuracy in clinical decision support during testing phases Enabled real-time clinical trial matching with biomarker filtering Developed a streamlined UX for oncologists with zero-code interaction Created a compliant and scalable system ready for hospital-level deployment

What we learned

Collaboration is key: Close work with clinicians, geneticists, and data engineers was crucial AI in healthcare must be explainable to gain adoption and regulatory approval Vector search and RAG are game-changers in grounding LLM outputs in scientific facts Fine-tuned models outperform generic LLMs in high-stakes medical scenarios

What's next for Doctor AI Agent

Add voice-based interaction using speech-to-text for live consultations Apply for clinical validation and regulatory approvals for pilot deployments in hospitals

Built With

Share this project:

Updates