ExoSense AI

Explainable machine learning for classifying NASA Kepler exoplanet candidates and false positives using leakage-safe feature engineering, ensemble models, and SHAP interpretation.

Comment

🪐 ExoSense AI — Finding Real Worlds in Starlight

💡 Inspiration

Astronomers can't photograph planets light-years away. Instead they stare at a star's brightness and wait for a tiny, repeating dip — the shadow of a planet crossing its star. The catch: most dips are lies. Eclipsing binary stars, instrument glitches, and light bleeding in from a neighbouring star all fake the same signal.

When I learned that NASA scientists spend enormous effort separating real planets from these impostors, I wanted to build an AI that does the same job — not on a toy dataset, but on real Kepler telescope data. The dream of pointing a model at the sky and having it whisper "that one's real" was impossible to resist.

🔭 What it does

ExoSense AI is a machine-learning pipeline that classifies 9,564 Kepler Objects of Interest (KOIs) from the NASA Exoplanet Archive into real exoplanets vs false positives / noise — using only the physical measurements of each detection, never the human verdict.

A planet's transit dip is set by simple physics — the depth is roughly the planet-to-star area ratio:

$$\delta \approx \left(\frac{R_p}{R_\star}\right)^2$$

So a "planet" that blocks too much light is almost certainly two stars eclipsing each other. ExoSense AI learns dozens of these tells at once.

🏆 Results

Framing Test F1 ROC-AUC Accuracy Cross-val F1
CONFIRMED vs FALSE POSITIVE 0.973 0.996 0.981 0.972 ± 0.002
CONFIRMED + CANDIDATE vs FALSE POSITIVE 0.901 0.970 0.903 0.903 ± 0.003

The cross-validation spread of just ±0.002 means the score is real generalisation, not a lucky split.

🛠️ How I built it

  1. Loaded the data safely — a robust loader that skips NASA's # comment lines and fails loudly on bad input.
  2. Killed data leakage first. The KOI table hides the answer in plain sight (koi_pdisposition, the koi_fpflag_* rejection flags, vetting comments, koi_score, and kepler_name — which only exists for confirmed planets). I removed all of them. This is the difference between a fake 100% and an honest 97%.
  3. Engineered domain features — log transforms for skewed quantities, transit geometry like duty cycle \( = T_{\text{dur}} / P \), signal-quality ratios (depth-per-SNR), stellar/planet interactions, observed-quarter counts, missing-value flags, and — my favourite — relative measurement error \( |\sigma_x| / |x| \), because a noisy, poorly-constrained fit is a strong hint of a false positive.
  4. Trained a ladder of models — Logistic Regression → Random Forest → tuned XGBoost / LightGBM / CatBoost → a stacking ensemble with a logistic-regression meta-learner that beat every individual model.
  5. Evaluated honestly — stratified train/validation/test split, decision-threshold tuning for F1, confusion matrix, ROC & PR curves, and cross-validated scores.
  6. Made it explainable — permutation importance + SHAP, plus a plain-English read-out.
pip install -r requirements.txt
jupyter notebook exoplanet_classification.ipynb   # seeded — reproduces every number

🧠 What the model learned

Permutation importance and SHAP agree, and it's beautiful: the model relies on exactly what astronomers check by hand — centroid offsets (koi_dikco_msky: if the dimming comes from an offset star, it's a false positive), planet radius, transit signal-to-noise, number of transits, and my engineered duty-cycle feature. It learned physics, not a shortcut.

For a non-technical friend: imagine a streetlight flickering. A real moth dims it briefly, regularly, by a tiny consistent amount — and the light clearly comes from the lamp. A passing headlight or a second nearby lamp can fake that flicker. ExoSense AI has seen thousands of real and fake flickers, and checks how deep and regular the dip is, whether the light truly comes from the target star, and whether the measurements are clean — then scores each candidate.

🧗 Challenges I ran into

  • Data leakage everywhere. My first model scored ~100% — a red flag. Tracking down every column that secretly encodes the NASA verdict (especially kepler_name, which only exists for confirmed planets) was the hardest and most important part.
  • The ambiguous CANDIDATE class. Candidates are genuinely undecided, which caps any model that must classify them. I solved this with a two-track design: an honest inclusive model and a headline CONFIRMED-vs-FALSE-POSITIVE model (how pro pipelines like NASA's Robovetter are actually trained).
  • Messy real data — entirely empty columns, a 32-bit "observed quarters" bitmask, mixed-type fields, and heavy-tailed distributions spanning many orders of magnitude. Tamed with log transforms and a clean leakage-safe preprocessing pipeline.
  • Compute — nested cross-validation with four boosters oversubscribed my CPU and stalled. Fixed by flattening parallelism.

🎉 What I'm proud of

A 0.973 F1 that is fully reproducible, leakage-free, and explainable — backed by cross-validation, a trained model artifact, and a clean public repo. Real NASA data, real science, real result.

🚀 What's next

Probability calibration, cost-sensitive thresholds (a missed planet ≠ a wasted follow-up), and validating across different Kepler observing quarters to prove the model transfers across instrument epochs — then pointing it at fresh TESS candidates.

Built With

Share this project:

Updates

Submission history