MedAssembly AI

Turn every hospital meeting into structured, actionable medical intelligence — powered by a collaborative system of specialized AI agents.

Comment

Inspiration: Closing the Clinical Gap

In modern healthcare, Multidisciplinary Team (MDT) meetings are the heartbeat of complex patient care. However, these critical sessions often suffer from the "Last Mile" data gap: vital clinical insights are shared verbally but remain trapped in unrecorded conversations or fragmented notes.

Physician burnout is at an all-time high, driven by administrative burden. I was inspired to build MedAssembly AI to solve this—not just by transcribing words, but by creating an autonomous "Digital Assembly" that listens, reasons, and synchronizes clinical consensus directly into the healthcare ecosystem.

Why this Hybrid Architecture? This system utilizes a high-performance split between Groq (Transcription) and PromptOpinion (Orchestration). We opted for this hybrid approach because:

  • Gemini Free Quotas: Standard free tiers for Gemini/OpenAI have restrictive rate limits (RPM/TPM) that prevent seamless, sub-second real-time transcription.
  • Model Constraints: While PromptOpinion leverages GPT-4o-mini for superior clinical reasoning, the platform does not currently support the gpt-4o-audio-preview model for direct audio input.
  • The Solution: By using Groq Whisper-v3 for ultra-low latency speech-to-text and PromptOpinion for agent-to-agent clinical synthesis, we achieve a production-grade experience that bypasses free-tier bottlenecks.

What it does

MedAssembly AI is a Multi-Agent Clinical Intelligence platform that transforms raw medical discourse into structured, interoperable data.

  • Active Listening: It captures real-time audio and routes it through a specialized agentic fleet.
  • Agentic Collaboration: A system of specialized AI agents (Cardiologist, Radiologist, Surgeon, Nurse) attend the meeting, each extracting data relevant to their field.
  • Consensus Logic: Using a custom A2A (Agent-to-Agent) layer, the agents interact to resolve conflicts and reach a unified clinical impression.
  • Standards-Ready: The final output is an exhaustive FHIR R4 Bundle, including Clinical Impressions, Care Plans, and mandated Tasks, ready for EHR integration.

How I built it: The Hybrid AI Pipeline

To achieve professional-grade reliability and ultra-low latency, I engineered a unique Hybrid Orchestration Architecture:

  • Groq & Whisper-v3: Used for lightning-fast Speech-to-Text. This overcomes the "latency wall" of standard cloud providers, enabling real-time feedback.
  • PromptOpinion Orchestrator: The "System Brain" that manages the A2A Protocol. It handles speaker diarization and role-specific reasoning.
  • SHARP Reasoning (Specialized Healthcare Agentic Reasoning Protocol): A custom context-management layer I implemented to ensure agents maintain focus on patient safety and clinical guidelines throughout the session.
  • Full-Stack Interoperability: Built with a high-performance .NET 8 backend and a Next.js 14 frontend, designed with a premium, dashboard-centric aesthetic for high-stress clinical environments.

Challenges I ran into: The Real-Time Pivot

The biggest hurdle was the "Data Velocity vs. Reasoning Depth" trade-off. Initially, using large multimodal models for direct audio processing was too slow for a fluid meeting environment and hit severe quota limits.

I strategically pivoted to a Modular Hybrid Pipeline: offloading the heavy transcription to Groq while preserving the "Deep Reasoning" for the specialized agents in PromptOpinion. I also had to build a custom buffering system to handle audio chunks in the background without losing context when switching between agent roles.

Accomplishments that I'm proud of

  • Solo MDT: Building a full ecosystem that mimics a multi-disciplinary team, from diarization to FHIR export, as a solo developer.
  • Technical Synergy: Successfully bridging A2A (collaboration), MCP (tooling), and FHIR (standards) into one seamless workflow.
  • Latency Mastery: Achieving a near-instant transition from "Voice" to "Actionable Insight," proving that agentic workflows can be fast enough for live clinical use.

What I learned

This project was a deep dive into the Interoperability of Intelligence. I learned that the future of Healthcare AI isn't a single "God Model," but a Network of Specialized Agents. Implementing the A2A protocol taught me how to manage "agentic consensus"—how to get different AI roles to agree on a patient's treatment plan while maintaining strict traceability.

What's next for MedAssembly AI

  • Ambient Clinical Presence: Direct integration with hospital IoT microphone arrays for truly "invisible" documentation.
  • Specialty Expansion: Training new agent roles for Metagenomics, Oncology, and Pediatric Metagenomic sequencing.
  • SMART on FHIR: Full production integration with Epic and Cerner environments.
  • Agentic Audit Trail: Implementing blockchain-based logging for agentic decisions to ensure 100% accountability in AI-assisted diagnosis.

Built With

  • .net-8
  • a2a-protocol
  • fhir-r4
  • groq
  • llama-3.3
  • mcp
  • next.js-14
  • prompt-opinion-platform
  • sharp
  • whisper-large-v3
Share this project:

Updates