FirstDoctor

A multilingual Generative AI clinical assistant that breaks language barriers, providing doctors with expert medical summaries and smart auto-prescriptions via QR triage.

Comment

Inspiration

In crowded clinics and hospitals, particularly in diverse regions, a massive communication gap exists. Patients often struggle to articulate their symptoms accurately due to language barriers, clinical anxiety, or medical illiteracy. They might say, "Mujhe 3 din se bukhar aur chhati mein dard hai," but fail to convey critical nuances.

This leads to a chain reaction: wasted consultation time, potential misdiagnosis, and severe doctor burnout. Traditional Hospital ERP systems are merely data-entry tools; they don't understand the patient. We were inspired to build FirstDoctor AI to act as an intelligent clinical bridge, ensuring that every patient is heard in their native language and every doctor receives world-class, structured medical insights instantly.

What it does

FirstDoctor AI is a complete, zero-friction Generative AI Hospital ERP ecosystem. It transforms chaotic OPDs into streamlined, paperless environments through a 5-step automated workflow:

  1. Smart QR Check-in & Native Voice AI: Patients scan a dynamically generated QR standee outside the clinic. Without downloading any app, they speak their symptoms into their phone in their native language (e.g., Hindi, Marathi, Gujarati, English).
  2. Instant Vitals Sync: At the reception, basic vitals (BP, Sugar) are logged and instantly synced to the patient's digital AI file.
  3. Expert Clinical Summary: As the patient walks into the cabin, the doctor doesn't see raw text. The AI has translated and structured the regional input into a highly precise History of Present Illness (HPI) and identified top differential diagnoses.
  4. One-Click Auto-Prescriptions: The AI pre-generates a master-level treatment plan, including targeted Investigations (Blood Tests/X-rays), exact Medications with dosages, and lifestyle precautions. The doctor simply reviews, edits if necessary, and clicks "Approve".
  5. Seamless Pharmacy Handoff: The approved digital prescription is instantly flashed at the medical store. No illegible handwriting, zero waiting time.

How we built it

We built FirstDoctor AI with a robust and scalable tech stack tailored for a B2B SaaS environment:

  • AI Engine: We leveraged the Google Gemini API for our core intelligence. We engineered a highly specific "100-Year Expert Chief Medical Officer" prompt that forces the LLM to output strictly formatted JSON containing the clinical summary and treatment plan.
  • Frontend: HTML5, Tailwind CSS for a futuristic, glassmorphism-inspired UI, and Vanilla JavaScript with the Web Speech API for real-time native voice-to-text transcription.
  • Backend & Database: PHP 8.x routing data securely to a MySQL (PDO) database.
  • Infrastructure: Live deployed on an OpenLiteSpeed/Apache server using advanced .htaccess configurations for clean URLs and strict .env file security to protect API keys. We also utilized HTML2Canvas and Base64 encoding to generate high-resolution downloadable QR standees dynamically.

Challenges we ran into

  • Controlling AI Hallucinations: When translating conversational native language into clinical terms, LLMs can sometimes hallucinate or provide markdown-heavy text. We had to heavily refine our prompt engineering to ensure the Gemini API returned strict, parseable JSON objects without markdown backticks, ensuring the PHP backend wouldn't crash.
  • Cross-Origin QR Generation: We faced CORS policy blocks when trying to download dynamically generated QR codes via the browser canvas. We solved this by using server-side PHP to fetch the QR image and embedding it as a Base64 string directly into the DOM before triggering the download.
  • Server Security: Securing environment variables on a live shared hosting environment required overriding default server behaviors with strict RewriteRule configurations to prevent direct access.

Accomplishments that we're proud of

  • Successfully transforming a fragmented hospital visit into a seamless 100% digital pipeline from the waiting room to the pharmacy.
  • Achieving incredibly high accuracy in translating casual regional speech into professional medical terminology.
  • Deploying a fully functional, live B2B SaaS MVP that hospitals can actually start using today with zero onboarding friction for patients (no app downloads).

What we learned

  • Prompt Engineering is a Science: Getting consistent, structured JSON out of a generative model requires rigorous constraint setting in the prompt.
  • UX is Everything in Healthcare: We learned that patients in distress won't use complex apps. The QR-to-Web approach proved to be the most accessible solution.
  • The Power of Gemini: We were amazed by how quickly and accurately the Gemini model could parse unstructured, grammatically incorrect regional text and deduce highly logical differential diagnoses.

What's next for FirstDoctor AI

  • Multimodal AI Integration: Allowing patients to snap a picture of their old medical reports or lab results during check-in, which the AI will parse and add to the doctor's summary.
  • WhatsApp Bot Integration: Moving the triage process to WhatsApp so patients can book appointments and explain symptoms from home before even reaching the hospital.
  • EHR Integration: Building APIs to seamlessly push our AI summaries into existing global Electronic Health Record (EHR) systems.

Built With

Share this project:

Updates

Submission history