ExoVet

An honest, explainable machine-learning model that vets NASA Kepler signals the way real astronomers do — no shortcuts, no data leakage.

Comment

Inspiration

When a planet crosses its star, it casts a faint shadow — a tiny dip in starlight. But most of these dips aren't planets at all; they're eclipsing binary stars, instrument noise, or unresolved signals. NASA's Kepler mission flagged thousands of them, far too many to check by hand. I wanted to build a model that could automate this vetting — but more importantly, one that does it honestly and can explain itself.

What it does

ExoVet classifies each Kepler signal as CONFIRMED (a real exoplanet), FALSE POSITIVE (not a planet), or CANDIDATE (unresolved). It reaches 87% accuracy and a 0.84 Macro F1 score (cross-validated 0.83) — and, crucially, it earns that score without cheating.

How I built it

Using ~9,564 real signals from the NASA Exoplanet Archive, I explored the data, then made the single most important decision of the project: removing "leaky" columns. Several columns (like koi_pdisposition, kepler_name, and the false-positive flags) secretly reveal the answer. Keeping them would inflate the score while teaching the model nothing real — like studying with the answer key. Removing them makes the model honest.

I then engineered 3 domain features (signal-per-transit, transit shape, planet-star size ratio) and built up through a Decision Tree (0.73) → Random Forest (0.82) → XGBoost (0.84) → a final stacking ensemble that combines Random Forest and XGBoost. I evaluated with Macro F1 (not accuracy, because the classes are imbalanced), cross-validation, a confusion matrix, ROC curves, and SHAP.

I also built an interactive explainer website so anyone — even non-technical visitors — can understand how the model thinks, in plain English.

Challenges I ran into

The hardest class is CANDIDATE — those signals are genuinely ambiguous, unresolved even in the original scientific pipeline. No model can cleanly separate data that's inherently uncertain. Learning to report this honestly, rather than chase a fake-perfect number, was a challenge in itself.

Accomplishments that I'm proud of

  • A model that's honest (no data leakage) — its 0.84 is real, not inflated.
  • SHAP explanations showing the model relies on the same clues astronomers use: centroid offset (did the light come from this star or a neighbor?), planet size, and signal strength.
  • An interactive website that makes the whole thing understandable to anyone.

What I learned

How to spot and prevent data leakage, why F1 beats accuracy on imbalanced data, how ensembles and cross-validation work, and how to turn a "black box" into an explainable story.

What's next

More careful feature selection, further tuning of the stacking model, and validation against additional data.

Built With

Share this project:

Updates