MedGuard AI

Problem Statement

Medical chatbots are increasingly used for symptom checking, triage, and patient education, but they often hallucinate—generating inaccurate, misleading, or even harmful medical advice. This can result in:

Incorrect self-diagnosis, leading to delayed medical attention
Unsafe treatment recommendations, putting patient health at risk
Regulatory violations (e.g., HIPAA, GDPR, FDA compliance failures)
Loss of trust in AI-powered healthcare tools

Developers currently rely on manual testing, which is slow, expensive, and prone to oversight. A more efficient solution is needed to automate hallucination detection and provide corrections.

Our Solution: MedGuard AI

MedGuard AI is an AI-powered hallucination detection platform that ensures medical chatbots provide factually accurate and reliable responses.

Simulates real-world patient questions and tests chatbot accuracy
Detects hallucinations by comparing chatbot responses to trusted medical knowledge
Flags unsafe or misleading responses and provides corrections
Seamlessly integrates into AI pipelines via API

Unlike generic AI evaluation tools, MedGuard AI is tailored for medical applications, serving as a critical safety layer for AI-driven healthcare tools.

How It Works

User submits a medical question → Sent to user-inputted chatbot using a websocket endpoint Chatbot response is analyzed → Processed using LangChain Fact-checking with FAISS → Compared against trusted sources (CDC, WHO, UpToDate) Hallucination detection → Flags responses that contradict medical guidelines Response correction & reporting → UI displays errors and suggests improved responses

Tech Stack

Backend: FastAPI + WebSockets Fact-Checking: LangChain + FAISS + Verified Medical Databases Frontend: Streamlit (for real-time monitoring & visualization)

Expected Impact

Significantly reduces AI hallucinations before chatbot deployment Saves hundreds of hours in manual chatbot testing Ensures compliance with medical safety regulations