Phantom Clinical Intelligence
Inspiration
Modern healthcare systems generate enormous amounts of structured patient data, yet clinical workflows remain largely reactive and encounter-driven. Physicians often have only a few minutes to review years of fragmented medical history spread across laboratory results, medications, chronic conditions, imaging, procedures, and preventative care gaps.
Most existing AI healthcare systems focus on:
- summarization,
- retrieval,
- or static risk prediction.
Very few systems attempt true longitudinal clinical reasoning — understanding how disease evolves across time, how multiple organ systems interact, and where intervention today could prevent deterioration years later.
We wanted to build a system capable of:
- reasoning over patient trajectories,
- identifying hidden deterioration pathways,
- modeling interconnected chronic disease cascades,
- and generating proactive clinical intelligence before the patient encounter even begins.
This led to the creation of Phantom Clinical Intelligence.
What It Does
Phantom Clinical Intelligence is a FHIR-native longitudinal reasoning platform that transforms fragmented patient records into structured, forward-looking clinical intelligence.
The system combines:
- MCP-native computational reasoning,
- longitudinal disease modeling,
- interoperable specialist-agent consultation,
- preventative care intelligence,
- and FHIR-aware orchestration
to generate clinician-ready pre-visit intelligence briefings.
The platform performs:
- longitudinal renal risk analysis,
- cardiovascular trajectory forecasting,
- metabolic syndrome progression reasoning,
- hepatic risk assessment,
- medication burden analysis,
- preventative care-gap detection,
- and specialist escalation when appropriate.
Instead of simply summarizing historical data, Phantom attempts to answer questions such as:
- Where is this patient heading over the next 3–5 years?
- Which intervention today prevents future deterioration?
- What hidden risks emerge from interacting chronic conditions?
- Which missing labs or preventative measures represent future danger?
System Architecture
Our architecture consists of three major components:
1. Phantom Nexus Agent
The Phantom Nexus Agent acts as the central orchestration layer configured within Prompt Opinion.
Responsibilities include:
- coordinating clinical workflows,
- invoking MCP tools,
- managing patient-model generation,
- escalating specialist consultation,
- and generating structured clinician-ready outputs.
The Nexus Agent acts as the bridge between Prompt Opinion, the MCP server, and specialist agents.
2. Phantom MCP Server
The Phantom MCP Server is the computational clinical intelligence engine of the platform.
It performs:
- FHIR retrieval,
- computational patient modeling,
- longitudinal disease forecasting,
- organ-system reasoning,
- intervention prioritization,
- and preventative intelligence generation.
The MCP server reasons across:
- renal systems,
- cardiovascular systems,
- metabolic systems,
- and hepatic systems.
We implemented custom longitudinal reasoning modules for each domain.
3. Phantom Specialist Intelligence Agent
The Phantom Specialist Intelligence Agent provides interoperable specialist consultation through the A2A protocol.
This component demonstrates:
- distributed reasoning,
- interoperable clinical agents,
- modular escalation architectures,
- and specialist-aware longitudinal consultation.
The specialist agent can independently analyze patient trajectories and return structured consultation intelligence back to the orchestrator workflow.
How We Built It
The system was built using:
- Python
- FastAPI
- Google ADK
- MCP protocol tooling
- A2A interoperability
- LiteLLM
- Gemini models
- SHARP/FHIR context propagation
We used:
- synthetic FHIR R4 patient data generated using Synthea™,
- Prompt Opinion for orchestration and deployment,
- ngrok for tunnel exposure during development,
- and custom middleware layers for interoperability handling.
MCP Integration
One of the core technical goals was building a truly patient-aware MCP system.
The MCP server:
- receives FHIR context,
- extracts patient-aware metadata,
- retrieves longitudinal patient records,
- assembles computational patient models,
- and exposes clinical reasoning capabilities through MCP tools.
This enabled the orchestrator agent to invoke:
- patient-model generation,
- intervention simulation,
- and longitudinal risk forecasting
through standardized MCP interfaces.
A2A Interoperability
A major architectural focus was interoperability between agents.
We implemented:
- custom A2A middleware,
- JSON-RPC response normalization,
- task-state correction,
- role normalization,
- and response shaping
to allow seamless communication between Prompt Opinion and external specialist agents.
This allowed the system to demonstrate modular specialist escalation workflows instead of relying on a monolithic architecture.
Longitudinal Clinical Intelligence
A major focus of the platform was systems-level longitudinal reasoning.
Instead of treating diseases independently, Phantom models interconnected deterioration cascades.
For example:
Obesity
→ Sleep Apnea
→ Hypertension
→ Accelerated CKD Risk
Metabolic Syndrome
→ Insulin Resistance
→ Hepatic Steatosis
→ MASLD Progression
NSAID Exposure
→ Nephrotoxic Burden
→ Progressive Renal Decline
This systems-level reasoning differentiates Phantom from traditional encounter summarization systems.
Challenges We Faced
This project involved several major technical challenges.
1. MCP + FHIR Context Propagation
One of the biggest challenges was ensuring FHIR context propagated correctly through Prompt Opinion into our MCP server.
We encountered issues involving:
- missing
x-fhir-server-urlheaders, - SHARP context extraction,
- and patient-aware tool invocation.
We implemented custom context extraction logic and middleware normalization to ensure patient context flowed correctly into the MCP reasoning layer.
2. A2A Interoperability
Interoperability between Prompt Opinion and external A2A agents required significant debugging.
We encountered:
- JSON-RPC schema mismatches,
- invalid task-state serialization,
- role normalization issues,
- and middleware translation failures.
We solved this by implementing custom A2A response-fix middleware and protocol translation layers.
3. Longitudinal Reasoning Complexity
Clinical longitudinal reasoning is significantly more difficult than static summarization.
We had to:
- design disease progression logic,
- normalize incomplete FHIR observations,
- handle missing laboratory values,
- model organ-system interactions,
- and generate clinically coherent intervention prioritization.
This required iterative refinement of computational reasoning modules across renal, cardiovascular, metabolic, and hepatic domains.
4. Infrastructure and Deployment
Running:
- MCP servers,
- orchestrator agents,
- external specialist agents,
- and ngrok tunnels
simultaneously introduced orchestration and deployment complexity.
We also encountered:
- timeout issues,
- Gemini free-tier rate limits,
- tunnel coordination challenges,
- and distributed debugging problems during integration.
What We Learned
This project taught us:
- how difficult longitudinal clinical reasoning truly is,
- how important interoperability standards are in healthcare AI,
- how MCP enables modular patient-aware tooling,
- and how agent architectures can coordinate specialized reasoning systems.
We also learned:
- the importance of robust middleware layers,
- how SHARP context propagation works,
- how FHIR-native systems differ from traditional AI pipelines,
- and how distributed agent ecosystems can support future healthcare infrastructure.
Future Directions
Potential future improvements include:
- oncology progression modeling,
- neurological deterioration forecasting,
- polypharmacy interaction intelligence,
- SMART-on-FHIR integration,
- persistent patient memory,
- evidence-grounded reasoning,
- real-time deterioration monitoring,
- and adaptive specialist escalation.
We also plan to explore:
- temporal patient graph modeling,
- confidence-calibrated clinical reasoning,
- and asynchronous distributed healthcare agent systems.
Final Thoughts
Phantom Clinical Intelligence explores how:
- longitudinal computational reasoning,
- interoperable agent systems,
- FHIR-native infrastructure,
- and preventative intelligence
can augment future healthcare decision-support workflows.
Rather than replacing clinicians, the goal is to reduce cognitive overload, surface hidden longitudinal risks, and provide proactive clinical intelligence that helps physicians intervene earlier and more effectively.
Log in or sign up for Devpost to join the conversation.