HealthGPT AI

Inspiration

Healthcare information is often fragmented and difficult for people to understand. A person may have symptoms but not know what they could be related to, receive a laboratory report filled with technical terms and numbers, or need help finding an appropriate healthcare provider. Usually, these tasks require using different platforms and understanding medical information that may not be easy for everyone to interpret.

We wanted to build a single platform that could bring these experiences together in a simple and accessible way. This led us to HealthGPT AI — an AI-assisted healthcare platform designed to help users understand their health information and navigate toward appropriate next steps, while clearly keeping the system educational rather than presenting it as a medical diagnosis.

What it does

HealthGPT AI provides several healthcare-assistance features through one platform:

Symptom Checker: Users can select symptoms they are experiencing, and a trained machine-learning model generates ranked candidate conditions that may be associated with the selected symptoms. The results are explicitly presented as model-generated candidates and not as a diagnosis.
Medical Report Analysis: Users can upload laboratory reports in JPG, JPEG, or PNG format. The system uses Tesseract OCR to extract text from the report and a rule-based parser to identify structured laboratory values, units, and reference ranges. Gemini then generates an easy-to-understand educational explanation of the extracted information.
AI Health Chat: Users can ask general health and medical-information questions through a conversational interface. Gemini is used to generate educational responses while the system maintains safety-oriented instructions and avoids presenting itself as a doctor.
Find Doctors: Users can discover nearby hospitals, clinics, and healthcare providers. The application can use the user's latitude and longitude or allow the user to search for another location.
Health History: Analyzed medical reports are associated with authenticated users and can be accessed later through their health history.
Authentication: The platform supports traditional email/password authentication as well as Google OAuth.

The goal is not to replace healthcare professionals, but to make health information easier to understand and help users identify appropriate next steps.

How we built it

We built HealthGPT AI as a full-stack application with a React-based frontend and a Python FastAPI backend.

The frontend was developed using React, TypeScript, Vite, Tailwind CSS, React Router, Axios, React Markdown, Recharts, Leaflet, and React-Leaflet. It provides the user interface for authentication, symptom selection, medical-report uploads, AI chat, healthcare discovery, and health history.

The backend was developed using Python and FastAPI. It provides versioned REST API endpoints for authentication, symptoms, predictions, medical reports, AI chat, and healthcare discovery.

For the symptom prediction system, we trained machine-learning models using a dataset containing 377 symptom features. The evaluated dataset contained 24,626 test samples. We experimented with Logistic Regression and XGBoost and compared their performance using Top-1, Top-3, Top-5 accuracy, precision, recall, and F1-score. Logistic Regression achieved a Top-1 accuracy of approximately 92.57% and a Top-5 accuracy of approximately 99.86% on our evaluation data, while XGBoost achieved approximately 91.96% Top-1 accuracy and 99.70% Top-5 accuracy.

For the production prediction workflow, we use the trained Logistic Regression model. Selected symptoms are converted into a binary feature vector using the same symptom vocabulary and feature ordering used during training. The model then produces ranked candidate conditions.

For medical reports, we implemented an OCR pipeline using Tesseract OCR. Uploaded report images are processed and the extracted text is passed through a rule-based parser that attempts to identify laboratory tests, values, units, and reference ranges. Because OCR can produce errors, the system treats extracted information as potentially imperfect and encourages verification against the original report.

Google Gemini is used for the generative AI components, including medical-report explanations and the general AI Health Chat. Carefully designed system instructions are used to keep these responses educational, avoid diagnosis, avoid medication prescriptions, and highlight situations where professional medical care may be appropriate.

For data persistence, we use PostgreSQL with SQLAlchemy. User accounts and analyzed medical reports are stored in the database, allowing authenticated users to access their health history.

Authentication is implemented using JWT-based authentication, with Google OAuth available as an additional sign-in method.

The backend is containerized using Docker. The frontend is deployed on Vercel, while the backend and PostgreSQL database are deployed using Render.

Challenges we ran into

One of the biggest challenges was making the medical-report OCR pipeline reliable. Laboratory reports can have different layouts, fonts, spacing, abbreviations, and table structures. Even when the original image is clear to a human, OCR can sometimes interpret characters incorrectly. For example, values and reference ranges can become mixed together or characters such as O, 0, %, and other symbols can be misread. We therefore had to test the OCR output independently and improve the rule-based parsing logic instead of assuming that the OCR text would always be perfectly structured.

Another challenge was separating actual laboratory results from reference ranges and surrounding report text. A simple number-extraction approach was not sufficient because medical reports contain many numbers that are not test results. We built parsing rules around known laboratory-test names and structured patterns to improve extraction.

Deployment also introduced several practical challenges. The local development environment and the Docker environment had different configurations, particularly around environment variables and system dependencies. Tesseract had to be installed inside the Docker image because the Python pytesseract package alone does not provide the Tesseract executable. We also had to correctly configure environment variables for PostgreSQL, JWT authentication, Gemini, Google OAuth, and CORS.

Another challenge was integrating the machine-learning model into the actual application rather than only evaluating it offline. We needed to ensure that the production feature vector used exactly the same symptom vocabulary and feature ordering as the training pipeline. The trained model and label encoder were therefore packaged with the backend and loaded by the prediction service.

We also encountered issues while connecting authentication-protected endpoints during testing. This helped us verify that the API correctly rejects missing or invalid authentication tokens and accepts valid JWT access tokens.

Finally, the use of free-tier infrastructure and APIs introduced performance limitations. Gemini responses can take noticeable time, and the deployed application can also be slower because the project currently relies on free-tier services. We treated this as an important real-world engineering constraint rather than hiding it.

Accomplishments that we're proud of

We are proud that we moved HealthGPT AI from an idea into a working full-stack deployed application rather than stopping at an ML notebook or prototype.

We successfully integrated multiple technologies into one healthcare platform:

A trained machine-learning disease prediction pipeline.
A production-facing Logistic Regression prediction service.
OCR-based medical-report processing using Tesseract.
Rule-based extraction of laboratory results.
Gemini-powered educational explanations.
Gemini-powered general health chat.
JWT authentication.
Google OAuth authentication.
PostgreSQL database persistence.
User-specific health history.
Location-based healthcare discovery.
Docker-based backend deployment.
A React and TypeScript frontend.
A FastAPI backend.
Separate cloud deployment of the frontend and backend.

We are particularly proud of the model evaluation work. We evaluated both Logistic Regression and XGBoost rather than assuming that a single model would be the best choice. Logistic Regression achieved a Top-1 accuracy of approximately 92.57% and a Top-5 accuracy of approximately 99.86% on our evaluation data, which gave us a strong baseline for the symptom-prediction component.

We are also proud of implementing safety-oriented boundaries throughout the AI features. The prediction system does not present its outputs as confirmed diagnoses, and the generative AI components are instructed not to diagnose users or prescribe medication. This was an important part of designing HealthGPT as a responsible healthcare-assistance platform.

Most importantly, we learned how different parts of an AI product — machine learning, OCR, generative AI, backend APIs, databases, authentication, frontend development, and deployment — need to work together to create a usable application.

What we learned

Building HealthGPT AI taught us that developing an AI application is very different from building an isolated machine-learning model.

We learned how important the complete data pipeline is. A model can have strong evaluation metrics, but the application still depends on correctly converting real user input into the exact feature representation expected by the model. This made us pay close attention to feature names, ordering, validation, label encoding, and model loading.

We also learned that OCR introduces a completely different type of uncertainty. Unlike structured datasets, real-world documents are messy. OCR output needs validation and post-processing before it can safely be used by another AI system.

Another major lesson was that generative AI needs carefully designed boundaries, especially in healthcare-related applications. Instead of simply sending user information to an LLM, we designed instructions that distinguish reported information from AI interpretation and prevent the system from presenting candidate conditions as diagnoses.

We also gained practical experience with authentication, OAuth, PostgreSQL, API design, Docker, environment variables, CORS, cloud deployment, and debugging production environments.

Finally, we learned that performance is an important part of product design. A technically functional AI system can still provide a poor user experience if responses take too long. Our current use of free-tier infrastructure and the free Gemini model helped us identify this limitation and gave us clear directions for future optimization.

What's next for HealthGPT AI

Our next goal is to make HealthGPT AI more accurate, faster, and more useful while maintaining its educational and safety-focused approach.

Some of the improvements we plan to explore are:

Better prediction models: Experiment with improved feature engineering, model architectures, data quality, and validation methods to improve the symptom-prediction system.
More comprehensive medical-test support: Expand the report-analysis pipeline to support a wider range of laboratory tests and report formats.
Improved OCR and document understanding: Use stronger document-processing techniques to handle complex laboratory reports, tables, different layouts, and noisy images more reliably.
Faster AI responses: Optimize prompts, model selection, caching, infrastructure, and API usage to reduce response latency.
Doctor appointment booking: Extend healthcare discovery into appointment scheduling so users can move from finding a healthcare provider toward taking an appropriate next step.
Medicine delivery integration: Explore integration with legitimate healthcare and pharmacy services where appropriate, while maintaining strong safety and regulatory considerations.
Mobile application: Build dedicated mobile applications alongside the existing web platform to make HealthGPT more accessible on smartphones.
Personalized health history: Improve the health-history system so users can better organize and understand their previous reports and interactions.
Multilingual healthcare assistance: Expand language support so more users can understand health information in their preferred language.
Improved scalability and reliability: Move beyond free-tier infrastructure as the platform grows and introduce more robust production infrastructure.
Responsible AI improvements: Continue improving safety checks, uncertainty communication, evaluation, and human-professional handoff so that HealthGPT remains an educational assistance tool rather than a replacement for qualified healthcare professionals.

Our long-term vision is to make HealthGPT AI a unified and accessible healthcare-information platform that helps people better understand their health information and navigate toward appropriate healthcare resources.

Built With

Updates

Chetan Mummani started this project — Sep 08, 2026 07:42 AM EDT

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.