Hunting Exoplanets with Machine Learning

A leakage-audited XGBoost that sorts NASA Kepler signals into confirmed planet, candidate, or false positive — 93% F1, fully explainable, and honest about what starlight alone can't reveal.

Comment

Inspiration

Most student projects use toy datasets. This one hands you real data from a real space telescope. NASA's Kepler mission watched ~200,000 stars for the faint dips in light that betray a planet crossing its star — but most of those dips are impostors. I wanted to build the same triage a NASA scientist does: is this signal a confirmed planet, an unconfirmed candidate, or a false positive? And I wanted to do it honestly — no cheating, and able to explain every call.

What it does

It classifies 9,564 Kepler Objects of Interest into CONFIRMED / CANDIDATE / FALSE POSITIVE straight from transit and stellar measurements. A tuned XGBoost model reaches a cross-validated macro-F1 of 0.928 (~99% recall on false positives), and every prediction is explainable — I can show exactly which measurements drove any verdict.

How I built it

Python + scikit-learn + XGBoost. I started with EDA (class imbalance, missing-data patterns, physics-driven distributions), then ran a leakage audit — removing columns that secretly encode the answer (like kepler_name, which only confirmed planets have, and koi_pdisposition, the pipeline's own verdict). I engineered domain features from transit physics — fractional measurement uncertainties, log transforms, and consistency ratios like depth_consistency (does the observed dip match the fitted planet size?). I compared four models, tuned XGBoost with a randomized search, validated with 5-fold cross-validation, and interpreted it with gain importance, permutation importance, and SHAP.

Challenges I ran into

The biggest was resisting easy, inflated scores. Several columns leak the answer; including them gives a fake ~99% accuracy that teaches the model nothing. Auditing and removing them — and measuring how much the Robovetter flags contribute instead of blindly using them — was the hardest and most important part.

Accomplishments that I'm proud of

A cross-validated macro-F1 of 0.928 that's honest and reproducible. But most of all: discovering why the model struggles where it does. Nearly all its errors are CANDIDATE-vs-CONFIRMED — and that's not a bug. The only real difference between them is whether humans have finished the follow-up, which no light curve can encode. The model surfaces its own limits instead of hiding them.

What I learned

How professional exoplanet vetting actually works, why data leakage is the silent killer of ML projects, and how to make a model explainable to someone with no ML background. Also that the most interesting result isn't the accuracy number — it's understanding the science behind the errors.

What's next

Fold in raw Kepler light curves (not just tabular features), test the model on the newer TESS mission's candidates, and turn the SHAP explanations into a simple tool that flags why any new signal looks like a false positive.

Built With

Share this project:

Updates