Mission Control AI

▶ Live demo: https://ubhacking-frontend-nine.vercel.app · Code: https://github.com/Jeetkavaiya/UBHacking (The backend runs on a free tier. If the header says "reconnecting", wait about 50 seconds while it wakes up.)

Inspiration

Spacecraft operators watch thousands of telemetry channels, and most monitoring tools only say something is wrong. A person still has to work out why, decide what to do, and check that it worked, often while flooded with false alarms. We wanted an AI flight controller that runs that whole loop, while humans stay in charge of anything risky.

What it does

Mission Control AI watches two sources at once:

Real NASA telemetry: the SMAP satellite and the Curiosity rover, replayed from NASA's public anomaly dataset. This proves the detector on real spacecraft data.
SIM-1, a simulated satellite we built: 11 named sensors (battery, solar array, heater, reaction wheel, radio). It orbits through sunlight and shadow, its parts affect each other, and our commands really change it. Every fault scenario has a hidden answer key. This proves the decisions.

For every reading, it:

Detects spikes, drifts and frozen sensors with a learned per-channel forecaster plus calibrated checks.
Groups one fault's symptoms into one incident (a worn reaction wheel alarms on 4 sensors but is 1 incident).
Diagnoses the incident with a Gemini flight director. It reads a case file (symptom timing, every sensor against its normal range, orbit, ops plan, past incidents), can call a tool to pull sensor history, and must name the root-cause part, the evidence, an action and a plan B.
Acts with bounded autonomy, enforced in code rather than by the AI:
- Low-risk actions run at once.
- Medium-risk actions get a 30-second veto countdown.
- Safe mode needs human approval.
Verifies the fix. It runs plan B if the first fix fails, and escalates to an engineer if that fails too.
Reports: every step is spoken with ElevenLabs and shown on a live 3D dashboard, and each incident ends with an after-action report that becomes memory for the next one.

How we built it

Backend: Python and FastAPI, streaming over WebSockets to a three.js dashboard.
Learned detector: for each channel, a ridge-regression forecaster predicts the next reading. It alarms when reality keeps disagreeing with the forecast, with Telemanom-style pruning. It runs as an ensemble with spike, drift and frozen checks.
SIM-1 simulator: sunlight, battery, thermostat and reaction-wheel physics, plus 6 fault scenarios with hidden answer keys.
Gemini 2.5 Flash: function calling plus a strict JSON schema. Rules take over automatically when Gemini is unavailable, and a quota guard keeps us inside the free tier.
ElevenLabs: text-to-speech with caching.
Hosting: the dashboard on Vercel, the backend on Render (Docker).
Tests: 45 automated tests with a fake Gemini, so testing costs no quota.

Challenges we ran into

NASA's data is anonymized: channels are codes and values are scaled, so no AI can say what broke. Our answer was the hybrid design: real data proves detection, and a simulator with named parts proves reasoning.
Too many false alarms: our first detector raised 143 false alarms. A learned forecaster, a persistence filter and pruning cut that to 35, while catching more real anomalies.
Gemini's free tier allows 20 requests a day. We designed around it: 2 requests per incident, plan B costs no extra request, and a daily cap persists across restarts.
Vultr: the hackathon's Vultr credit activation failed. The organizers asked teams to describe how they would have used it, so we deployed on Render instead (see What's next).

Accomplishments that we're proud of

Root cause: Gemini 5/6 vs. rules 3/6 on SIM-1, graded automatically against hidden answer keys.
- It recognized a planned imaging pass in eclipse as not a fault.
- It traced a draining battery to a failing solar array ("0.60 A vs 2.27 A expected in full sunlight").
- It skipped a useless reboot on a dead sensor.
- Its one miss: a stuck heater, which it blamed on the sensor that controls the heater.
Honest detection on all 81 real NASA channels (105 labeled anomalies):
- Precision went from 33.2% to 67.6%, recall from 67.6% to 69.5%, and false alarms from 143 to 35.
- NASA's own LSTM (Telemanom) reaches 87.5% / 80%. We publish the full tuning sweep.
A full closed loop: detect, diagnose, act safely, verify, run plan B, report, remember.

What we learned

Detecting an anomaly is the easy part. The value lies in context (orbit, ops plan, how parts connect), in verifying that fixes worked, and in keeping autonomy bounded so that people stay in control. An honest evaluation also beat a pretty number: measuring 143 false alarms is what pushed us to build the learned detector.

What's next: including how we'd use Vultr

Vultr Cloud Compute: run our Docker image always-on, with HTTPS, so there's no free-tier cold start.
Vultr Cloud GPU: train a true LSTM forecaster per channel (Telemanom-style) to close the precision gap.
Vultr Object Storage: host the full 425 MB NASA dataset, so the live demo can switch among all 81 channels.
Vultr Managed PostgreSQL: persist incident memory across restarts. The app already writes to Postgres through DATABASE_URL.
Run more Gemini trials per scenario, and add more fault scenarios.

Disclosures

Data: real NASA SMAP/MSL telemetry from the public Telemanom dataset (Hundman et al., 2018), replayed faster than real time and anonymized. Only P = Power and R = Radiation are documented.
Simulated: SIM-1 is a fictional satellite we built during the event, and it is labeled "Simulated" everywhere. Injected faults are labeled. Commands on NASA data are simulated.
Demo channel selection: the live demo uses a curated set of 10 NASA channels (M-6 and S-2 excluded for excessive false alarms). Precision and recall are reported on all channels.
Existing work used: open-source libraries, the NASA dataset, and the Telemanom paper as a method reference and comparison point only. NASA's pretrained models are not our detector.
Hosting: Render instead of Vultr, because Vultr credits could not be activated.
Built during the hackathon: all code, written with the help of an AI coding assistant (Claude Code).