🏥 SymptomSense: AI-Powered Medical Diagnostic Assistant

Building Trust in AI Healthcare Through Multi-Agent Intelligence and Confidence-Aware Decision Making

💡 Inspiration

Healthcare accessibility remains one of humanity's greatest challenges. We've all experienced it—waiting hours in emergency rooms for preliminary assessments, struggling to interpret medical information online, or wondering if symptoms warrant immediate attention. The COVID-19 pandemic amplified these issues, with telemedicine usage increasing by 154% in 2020, yet most digital health tools remain simplistic chatbots that either provide generic advice or dangerously overconfident diagnoses.

The problem: Existing AI health assistants fall into two camps—overly cautious systems that redirect everything to "consult your doctor," or reckless ones that provide confident diagnoses without acknowledging uncertainty.

Our vision: What if AI could be intelligently uncertain? What if a system could analyze your symptoms across multiple modalities, transparently show its confidence level, and know exactly when to escalate to human experts?

We were inspired by recent advances in agentic AI architectures (LangGraph, AutoGPT) and retrieval-augmented generation (RAG) that enable AI systems to reason with external knowledge. We asked: "Can we build a medical assistant that doesn't just answer questions, but orchestrates multiple specialized agents, evaluates its own confidence, and prioritizes patient safety?"

SymptomSense was born from this question—a multi-agent medical diagnostic platform that combines computer vision, medical literature retrieval, real-time web research, and confidence-aware decision-making into a cohesive, trustworthy system.

🎯 What It Does

SymptomSense is an intelligent medical diagnostic assistant that analyzes text, images, and voice inputs through a coordinated network of specialized AI agents to provide evidence-based health insights with transparent confidence scoring.

Core Capabilities

🔬 Multi-Modal Input Processing

Text Queries: Natural language symptom descriptions, medical questions
Image Analysis: Upload chest X-rays, skin lesions, or other medical images
Voice Input: Speech-to-text powered by Faster Whisper for hands-free interaction

🤖 Agentic AI Architecture

SymptomSense doesn't use a single AI model—it orchestrates 6 specialized agents via LangGraph:

Input Classification Agent: Routes queries to appropriate analysis pathways
Vision Analysis Agent: ViT (Vision Transformer) trained on medical images for chest X-ray classification (NORMAL/PNEUMONIA detection)
RAG Document Agent: Retrieves relevant passages from curated medical literature using vector similarity search
Web Research Agent: Brave Search integration for latest clinical guidelines and temporal medical information
Synthesis Agent: Gemini 2.5 Flash combines insights from all sources into coherent responses
Confidence Verification Agent: Meta-analysis of AI certainty across all modalities

📊 Transparent Confidence Scoring

Every response includes a multi-dimensional confidence profile:

Overall Confidence: High (>80%), Medium (60-80%), Low (<60%)
Image Analysis Confidence: Computer vision model certainty
Knowledge Base Confidence: RAG retrieval relevance scores
LLM Reasoning Confidence: Language model logprob analysis

⚠️ Human-in-the-Loop (HITL) Safety

Low-confidence cases (<70%) are automatically flagged for medical professional review and added to a review queue with:

Original query and analysis
Confidence breakdown
Suggested expert specialties
Priority level

🌐 Real-Time Knowledge Updates

Unlike static medical chatbots, SymptomSense queries live web sources when:

Temporal queries detected ("latest 2025 treatment guidelines")
Local knowledge base has low confidence (<50%)
User explicitly requests recent information

📚 Evidence-Based Responses

All answers include:

Citations from medical literature with relevance scores
Links to clinical practice guidelines from authoritative sources (NCBI, NIH, medical journals)
Warnings when information is uncertain or requires specialist confirmation