Medysinc

Revolutionizing triage with an interactive 3D body map. Our AI analyzes symptoms visually to provide instant, accurate medical guidance and personalized care plans.

Comment

MediSync: AI-Powered 3D Triage & Health Ecosystem

Inspiration

The global healthcare system faces a critical challenge: the disconnect between patient symptoms and timely, accurate diagnosis. Emergency rooms are overcrowded with non-urgent cases, while serious conditions often go undetected until it's too late. We were inspired by the potential of combining objective data (from wearables) with subjective patient experiences (symptoms) to create a smarter, more accessible triage system.

Originally conceived during a local hackathon to solve insurance coverage gaps, we realized the potential was much broader. We wanted to build a tool that empowers users to understand their health through an intuitive 3D interface and AI-driven semiology, democratizing access to preliminary medical assessment.

What it does

MediSync is a dual-platform ecosystem designed to bridge the gap between patients and healthcare providers:

  1. MediSync Mobile App (Android):

    • Acts as the patient's comprehensive digital health passport.
    • Data Aggregation: Users can upload their full clinical history, medical photos (e.g., skin lesions, previous X-rays), and lab results.
    • Wearable Integration: Seamlessly connects with wearables to collect real-time vital signs and health metrics.
    • Secure Storage: Stores all this sensitive user profile data securely to inform the triage process.

  2. AnatomIA 3D Triage (Web Platform):

    • An interactive 3D Human Model visualization where users can pinpoint exactly where they feel pain or discomfort.
    • AI-Powered Semiology: Utilizing advanced LLMs (Gemini/Claude), the system conducts a medical interview (anamnesis) based on the selected body part and the user's data from the mobile app.
    • Provides a preliminary triage assessment, suggesting urgency levels and potential conditions, guiding the user on whether to see a doctor immediately or monitor symptoms at home.

How we built it

We built MediSync using a modern, hybrid tech stack to ensure performance and scalability:

  • Mobile: Native Android development using Kotlin for robust performance and seamless integration with wearable APIs.
  • Web/3D: Built with React and Vite. The 3D visualization is powered by Three.js, allowing for a lightweight, interactive human model in the browser.
  • AI & Backend:
    • Google Gemini & Anthropic Claude: We orchestrated these models to handle the medical dialogue, ensuring the AI asks relevant semiological questions (onset, duration, intensity) based on the specific anatomical region selected.
    • AWS Lambda: Serverless functions to handle the API requests and orchestrate the communication between the frontend and the AI models.
    • Node.js: For backend proxy services.

Challenges we ran into

  • 3D & Web Integration: Mapping 3D coordinates on a model to specific medical anatomical regions was complex. We had to ensure that clicking on the "chest" triggered the correct cardiac or respiratory context for the AI.
  • AI Hallucinations & Safety: Ensuring the AI provides medically relevant information without making dangerous diagnoses. We spent significant time on Prompt Engineering and implementing guardrails to keep the AI in a "triage" role rather than a "diagnostic" role.
  • Data Synchronization: Creating a seamless flow where data collected in the mobile app (like heart rate) informs the web-based triage session.

Accomplishments that we're proud of

  • Successfully integrating a 3D interactive model with a conversational AI. It feels like magic when you click a body part and the AI immediately knows the context.
  • Creating a functional prototype that covers both the data collection (Mobile) and the intelligent analysis (Web).
  • Implementing a robust "Flow Control" system for the AI to ensure it follows clinical protocols during the interview.

What we learned

  • User Experience is Critical in Health: A 3D model is much more intuitive for patients to describe pain than a text box.
  • The Power of Context: AI performs significantly better when provided with structured context (wearable data + specific anatomical location) rather than open-ended chat.
  • Interoperability: The importance of standardizing health data for effective use in AI models.

What's next for MediSync

  • Telemedicine Integration: Allowing users to share their generated triage report directly with a doctor via video call within the app.
  • Mental Health Module: Expanding the AI to detect non-physical symptoms and offer mental health triage.
  • Blockchain Security: Implementing decentralized storage for patient records to ensure absolute privacy and ownership of medical data.
  • Partnerships: Piloting the system with local clinics to reduce triage wait times.

Built With

Share this project:

Updates