Consultify

• 

  Consultify Banner

• 

  Multilingual Consultation

• 

  Prescription Assistance

Inspiration

Healthcare in Africa faces critical accessibility challenges. Millions of lives are lost because patients cannot effectively communicate their symptoms to healthcare providers. With 1,500 to 3,000 languages spoken across Africa, language barriers between patients and healthcare providers create significant obstacles to quality care.

How do we solve this?

Retrieval Augmented Generation (RAG) agents can provide context-aware understanding of patient symptoms and make informed decisions about specialist referrals. Additionally, modern LLM models excel at multilingual tasks. Combined with structured outputs, LLMs can generate context-aware translations that bridge communication gaps.

What it does

Consultify revolutionizes telemedicine by enabling context-aware RAG and multilingual communication. The platform is powered by TiDB Serverless on TiDB Cloud, which provides the vector store infrastructure.

Understand Symptoms - Users describe their symptoms to an intelligent clerking system with vectorized access to the open-access textbook "Microbiology, Pharmacology, and Immunology for Pre-Clinical Students."

Doctor Selection - Based on the agent's understanding, it selects the most suitable specialist from our database of healthcare providers.

Multilingual Consultation - Using Mistral Large through Amazon Bedrock with structured outputs, the platform provides context-aware translations, allowing each party to communicate in their native language.

Prescription Assistance - After diagnosis, my medical presciption agent analyses patient's symptoms and doctor converstaion to recommend prescriptions using RAG with vectorized access to the British National Formulary for medication guidiance.

With Consultify, we achieve:

• The simplest onboarding experience in telemedicine
• Seamless multilingual communication between patients and doctors regardless of language
• Prescription assistance for doctors prescribing medications

How we built it

Despite the fast-paced nature of this hackathon, I went the extra mile on all technical levels by using cutting-edge technology stacks and SDKs while maintaining production-grade coding standards and project architecture.

Agent Framework: I used LangChain and LangGraph for creating workflows and LLM interactions.

Vector Store: I implemented TiDB's vector store through LangChain integration.

Frontend: I built a modern React-based frontend using Next.js 15 with app-directory routing, Magic UI components, and Next.js API routes for database interactions.

ORM: Prisma served as the ORM layer.

Relational Database: TiDB's MySQL database stores structured data (patients, doctors, and consultations).

LLM: Mistral Large via Amazon Bedrock powers inference throughout the application. Mistral Embed serves as the embedding model for vectorizing the British National Formulary and the medical textbook.

Data Sources: Our primary data sources include:

• The British National Formulary: The authoritative UK reference guide providing healthcare professionals with evidence-based information on drug selection, dosing, interactions, and prescribing guidance.

• Microbiology, Pharmacology, and Immunology for Pre-Clinical Students: A peer-reviewed open textbook designed to support undergraduate medical education and pre-clerkship training.

Together, these resources enable context-aware consultations, allowing the clerk agent to make appropriate specialist selections and prescription recommendations.

Challenges we ran into

Workflow Development Challenges:

• Understanding LangGraph, particularly the routing mechanisms
• As a JavaScript developer, this was my first experience with a graph framework like LangGraph and implementing structured outputs

TiDB Integration Challenges:

• Initially encountered an "Insecure transport not allowed" error when using TiDB with Prisma, despite following the official Next.js example (which uses Drizzle)
• Resolved by appending ?ssl=true&sslaccept=strict to the connection string

Accomplishments that we're proud of

• Developed a novel telemedicine primitive enabling cross-language patient-doctor communication
• Created the first multilingual telemedicine platform with vectorized access to industry-standard medical knowledge bases
• Successfully overcame all technical challenges during development

What we learned

• How to develop and integrate retrieval augmented generation systems into AI applications
• Building intelligent workflows with LangGraph
• Understanding the distinctions between agents and workflows

What's next for Consultify

• Expand the medical knowledge base with additional pharmacology textbooks
• Optimize graph performance to reduce average invocation time
• Develop a mobile application for a mobile-first multilingual telemedicine experience

Built With

• flask
• langchain
• langgraph
• nextjs
• tidb