Afya diary

An interoperable healthcare companion that finds nearby services, fetches FHIR patient data, and delivers real-time health assessments through Prompt Opinion.

Comment

AFYA Diary — Your Personal Healthcare Companion

AFYA Diary Banner

An interoperable healthcare AI agent that finds nearby medical services, fetches real FHIR patient data, and delivers health assessments — all through Prompt Opinion's A2A platform.

🎥 Demo Video

Watch the demo on YouTube

📋 Table of Contents

Inspiration

Across the world, people from all walks of life struggle to navigate complex healthcare systems — finding a nearby clinic that's actually open, understanding what their prescription means, or knowing which specialist to see next. Healthcare information is scattered, disconnected, and hard to access when you need it most.

We've seen this firsthand — family members spending hours on the phone chasing appointments, patients leaving consultations confused about their own conditions, and communities underserved by fragmented digital health tools. Existing healthcare chatbots operate in silos. They can't access real patient data. They don't collaborate with other AI agents.

We built AFYA Diary to show that by combining A2A, MCP, and FHIR, we can create agents that actually work together — making healthcare guidance accessible, interoperable, and genuinely useful for everyone.

What it does

AFYA Diary is a multimodal healthcare AI agent running inside the Prompt Opinion platform. It provides three core skills:

Skill What It Does Standards Used
🏥 FindHealthServices Finds nearby hospitals, clinics, pharmacies, and mental health centers using GPS Google Maps API, MCP
🩺 GetPatientSummary Fetches patient demographics from FHIR servers using SHARP context FHIR R4, A2A SHARP Extension
📋 HealthAssessment Combines patient info, active conditions, and current medications into one overview FHIR R4, A2A, MCP

Every response includes: "I am not a doctor. This is not medical advice. Please consult a healthcare professional."

AFYA Skills Demo

How we built it

The Stack

┌─────────────────────────────────────────────────────────┐
│                   Prompt Opinion Platform                │
│  • A2A Agent Discovery                                  │
│  • SHARP FHIR Context Propagation                       │
│  • Agent Card Parsing                                   │
└─────────────────────┬───────────────────────────────────┘
                      │ A2A JSON-RPC + SHARP Headers
                      ▼
┌─────────────────────────────────────────────────────────┐
│              Google Cloud Run (FastAPI)                  │
│  • /mcp/sse — MCP Server Endpoint                       │
│  • /health/nearby-services — Hospital Finder            │
│  • /health/patient/summary — FHIR Patient Data          │
│  • /health/assessment — Full Health Overview            │
│  • /ws/{session_id} — Real-time Voice AI                │
└──────┬────────────────────┬─────────────────┬───────────┘
       │                    │                 │
       ▼                    ▼                 ▼
┌──────────┐    ┌──────────────┐    ┌──────────────┐
│  Google  │    │  HAPI FHIR   │    │  Gemini Live │
│   Maps   │    │   Server     │    │     API      │
└──────────┘    └──────────────┘    └──────────────┘

Key Technical Decisions

  1. MCP Server integrated directly into FastAPI: Instead of deploying a separate service, I mounted the MCP SSE transport onto the existing FastAPI app at /mcp. Single deploy, single service, lower costs.

  2. A2A Agent Card with FHIR Extension: The agent card at /.well-known/agent-card.json declares all three skills with proper JSON schemas and SMART scopes (patient/Patient.rs, patient/Condition.rs, patient/MedicationRequest.rs). Prompt Opinion discovers everything automatically.

  3. SHARP Context Ready: All health endpoints accept X-Patient-ID, X-FHIR-Server-URL, and X-FHIR-Access-Token headers. The platform injects these into every A2A message.

  4. Gemini Live for Voice: Real-time voice interaction via WebSocket, with a 5-state agent machine (IDLE → LISTENING → PROCESSING → NAVIGATING → COACHING) and barge-in support.

Architecture Diagram

Challenges we ran into

1. Agent Card Schema Iterations

Problem: Prompt Opinion's A2A parser required specific fields — id, tags on skills, protocolBinding, protocolVersion on interfaces, and the capabilities.extensions structure for FHIR context.

Solution: Six iterations of the agent card, each time reading the exact error message and adjusting. The final version nests the FHIR extension inside capabilities.extensions with uri (not url) and params.scopes.

2. FHIR Extension Recognition

Problem: The extensions field at the top level of the agent card was ignored by Prompt Opinion. Moving it inside capabilities caused a type mismatch because the parser expected a list.

Solution: Following Prompt Opinion's exact documentation format — capabilities.extensions as an array with uri, required, and params.scopes — resolved the issue. The extensions tab now correctly shows the FHIR context toggle.

3. WebSocket Keepalive

Problem: Active voice sessions were being killed by the heartbeat monitor because audio chunks didn't update the heartbeat timestamp.

Solution: Modified update_heartbeat() to be called on every received message (text AND binary), and changed the heartbeat loop to only kill connections where the ping send itself fails — not time-based.

Accomplishments that we're proud of

  • ✅ Full A2A + MCP + FHIR Stack: All three standards working together in one codebase, deployed as a single Cloud Run service.

  • ✅ Auto-Discovery: Prompt Opinion reads the agent card and automatically discovers all three skills — no manual configuration needed.

  • ✅ SHARP Context Integration: Patient data flows from Prompt Opinion through to the FHIR server without any custom authentication code on our side. The platform handles token propagation.

  • ✅ Real-time Voice AI: Gemini Live integration with sub-200ms response latency and intelligent barge-in handling. AFYA listens, speaks, and can be interrupted naturally.

  • ✅ Safety First: Medical disclaimer on every response, emergency keyword detection, and a SOS trigger with GPS location via SMS.

  • ✅ Single Deploy: MCP server, REST API, and WebSocket all live in one Cloud Run service. Simple, cost-effective, easy to maintain.

What we learned

  1. Standards eliminate glue code. If Prompt Opinion didn't natively support A2A + SHARP, we would have spent weeks building authentication, context propagation, and agent discovery. Instead, we focused entirely on the healthcare features.

  2. Agent cards are the key to interoperability. A well-defined agent card is like a REST API's OpenAPI spec — it allows discovery without any prior coordination between systems.

  3. FHIR is more than a data format. The SMART scopes model (patient/Patient.rs, patient/Condition.rs) provides a precise permission system. It's not just about accessing data — it's about accessing the right data with user consent.

  4. Voice AI in healthcare needs guardrails. Every response must include a disclaimer. Emergency keywords must trigger escalation immediately. Safety can't be an afterthought.

  5. Cloud Run is ideal for agent hosting. Serverless, auto-scaling, WebSocket support, and one-command deploy. Perfect for hackathon velocity and production reliability.

What's next for AFYA Diary

Feature Description
🌍 Multi-language Support Leverage Gemini's multilingual capability for Swahili, French, Arabic, and more
📱 Native Mobile Apps React Native apps with offline detection for low-connectivity areas
🏥 Hospital Indoor Navigation Guide patients through hospital corridors to their appointments
💊 Medication Reminders Push notifications for medication schedules pulled from FHIR
🔐 OAuth Authentication User accounts with personal emergency contacts and session history
🤝 Multi-Agent Workflows AFYA Diary collaborating with specialist agents (cardiology, pediatrics, mental health) via A2A
📊 Health Trend Dashboard Long-term tracking of health check-ins with visual analytics

Built With

Category Technologies
AI / LLM Gemini Live API, Google GenAI SDK
Backend Python, FastAPI, WebSocket, MCP SDK
Cloud Google Cloud Run, Cloud Firestore, Artifact Registry
Standards A2A, MCP, FHIR R4, SHARP
Maps Google Maps API (Places, Directions, Geocoding)
Frontend Next.js 14, TypeScript, Tailwind CSS
Infrastructure Docker, Terraform
Platform Prompt Opinion

Try it yourself

  1. Clone the repository: git clone https://github.com/spheresolutiondevelopers/Afya-Diary-Agent.git
  2. Install dependencies: pip install -r requirements.txt
  3. Set up .env with your Gemini and Maps API keys
  4. Run locally: python -m app.main
  5. Deploy: gcloud run deploy afya-backend --source . --region=us-central1 --allow-unauthenticated
  6. Add to Prompt Opinion: External Agents → Add Connection → Agent Card URL: https://your-url.run.app/.well-known/agent-card.json
  7. Or try it directly on the Prompt Opinion Marketplace

Built with ❤️ for the Agents Assemble - Healthcare AI Endgame Hackathon

Built With

Share this project:

Updates