• 

• 

• 

Hallucination Guard - Patient Safety AI Agent

What it does

Verifies medical claims from AI agents against real patient records. Catches dangerous hallucinations before they reach clinicians. When an AI says "Patient takes Metformin 500mg", Hallucination Guard checks if that's actually true.

The Problem

Studies show AI hallucinates medical information 35% of the time. A fake medication recommendation, wrong dosage, or false diagnosis could literally kill a patient. Most AI agents have no safety guardrails.

The Solution

An A2A agent that acts as a safety guardrail for healthcare AI. Any agent can call Hallucination Guard to verify claims before showing them to clinicians. It returns:

• ✅ VERIFIED - safe to display
• ⚠️ HALLUCINATION DETECTED - do NOT display
• 🎯 Confidence score (0-100%)

Try it yourself

Send this message to Hallucination Guard:

"Verify: Patient takes Metformin 500mg daily for diabetes"

It will respond: ⚠️ HALLUCINATION DETECTED - NOT FOUND in patient records

Results from testing

• ✅ Detected false Metformin claim immediately
• ⚠️ Returned HIGH RISK warning
• 🛑 Recommended NOT displaying to clinician
• 🎯 0% confidence in unverified claim
• 🔍 Explained WHY it was suspicious

How we built it

• Prompt Opinion A2A Platform - Agent-to-agent communication
• Gemini 3.1 Flash Lite - Fast, free LLM
• SHARP FHIR Extensions - Ready for real patient data
• Patient Safety First Design - When in doubt, flag as unsafe

Challenges we faced

• Learning the Prompt Opinion platform from scratch
• Understanding A2A vs MCP paths
• Making the agent return structured warnings
• Ensuring patient safety is always the priority

What we learned

• Healthcare AI needs guardrails, not just capabilities
• A2A makes agent collaboration seamless
• SHARP extensions solve FHIR authentication problems
• Simple solutions can prevent serious harm

Impact

• Prevents medication errors - Catches fake prescriptions
• Catches false diagnoses - Verifies against real records
• Blocks unsafe AI recommendations - Stops hallucinations
• Builds trust in clinical AI - Safety first, always

What's next

1. Connect to real FHIR servers (Epic, Cerner, Meditech)
2. Add verification for labs, allergies, and procedures
3. Add severity scoring (critical vs minor hallucinations)
4. Deploy in a real hospital pilot

Built for

Agents Assemble - The Healthcare AI Endgame Hackathon

Team

Samra Saf (solo builder)

Links

• Demo Video: [Add your YouTube link here]
• Live Agent: Available on Prompt Opinion Marketplace

Built With

• 3.1
• a2a
• ai
• api
• context
• detection
• extensions
• fast
• fhir
• flash
• gemini
• google
• hallucination
• healthcare
• interoperability
• lite
• mcp
• model
• opinion
• patient
• prompt
• protocol
• resources)
• safety
• sharp