I started with a simple question: if a model tells someone they have an 80% chance of heart disease, how do we know that number means anything?
Most projects I saw on this dataset stopped at accuracy. A model gets 73%, and that's the end of the story. But in a medical setting, accuracy hides the things that matter: how many sick people were missed, whether the probabilities are trustworthy, and why the model said what it said. I wanted to build something that answers those questions instead of skipping them.
What I built
CardioRisk AI takes 11 routine inputs (age, blood pressure, cholesterol, glucose, weight, lifestyle habits and so on) and returns a calibrated probability of cardiovascular disease. It also flags whether that probability crosses a screening threshold, and shows a chart of what pushed the prediction up or down. Behind that, there's a full analytics dashboard: ROC curves, cross-validation, confidence intervals, calibration and subgroup results.
How it went
I began with the usual approach: clean the data, train Logistic Regression, Random Forest and XGBoost, and compare them. They all landed within about one point of each other on ROC-AUC (79.6% to 80.6%). That was my first lesson. The "best" model barely mattered, so I stopped chasing a winner and started reporting confidence intervals instead.
The bigger surprise was the default threshold of 0.50. At that setting, the model missed roughly one in three patients. For a screening tool, that's the wrong trade. So I chose a lower threshold, 0.381, from out-of-fold training predictions so the test set stayed untouched. Recall went up to 80.4%, and precision dropped to 68.6%. More cases caught, more false alarms. I decided not to hide that cost. The app says it out loud.
I also removed a feature I'd engineered (age groups), because it was just a repackaged version of age, and I checked that calibration and tuning actually helped before keeping them, rather than assuming they did.
Challenges
The hardest part wasn't the modeling. It was staying honest: not leaking test data into the threshold, not calling a 0.1-point gain a win, and admitting that the dataset doesn't even say which gender code is female or male, so the app shows raw codes rather than guessing.
What I learned
In a medical setting, a good model isn't just one with a high score. It's one that tells you how sure it is, why it thinks so, and where it's likely to be wrong.
What's next
External validation on other datasets, and feedback from clinicians on whether the explanations are actually useful to them.
Important: CardioRisk AI is a research prototype trained on public data. It is not a medical diagnosis tool.
Built With
- data-science
- github
- healthcare
- joblib
- jupyter-notebook
- kaggle
- kagglehub
- machine-learning
- numpy
- pandas
- python
- scikit-learn
- streamlit
- xgboost
Log in or sign up for Devpost to join the conversation.