Penumbra

Penumbra: verified AI mission ops for a 24-satellite fleet. When power faults hit, the AI proposes, physics verifies, and a human approves. Nothing is uplinked that the simulator hasn’t survived.TBH

Comment

Inspiration

Satellite operators fly dozens of spacecraft with a handful of people and a few minutes of ground contact per orbit. A power fault that starts in eclipse can drain a battery before anyone sees the telemetry. LLM agents are good at triage and explanation, but nobody should uplink a command because a language model sounded confident. We wanted an agent that is allowed to be wrong, inside a system where being wrong cannot hurt the fleet: AI proposes, physics verifies, humans approve.

What it does

Penumbra is mission control for a 24-satellite LEO constellation during a power-fault storm.

  • Sense. Real orbits (CelesTrak elements), real Sun geometry (JPL DE421), real 10° contact windows at three polar stations. Telemetry is downlinked only inside passes; operators are blind in between.
  • Understand. A detector raises alerts; a scorer ranks six hypotheses (nominal, array degradation, stuck-on load, capacity fade, sensor bias, beta season) and compares each satellite with its peers to separate a fleet-wide environmental cause from a single-satellite hardware fault. A Gemini agent reads the alert, pulls telemetry, scores hypotheses, can run a short diagnostic test inside the current pass, and proposes recovery plans through a strict tool allowlist.
  • Act. Every plan is simulated by a Monte Carlo verifier under every plausible hypothesis before it can be scheduled. Plans that keep the battery above the floor under all of them pass; the best one is ranked; pre-approved low-risk plans execute under a named flight rule (Tier 1); everything else waits for the Flight Director's click (Tier 2). If nothing passes, the agent says so and recommends SAFE_MODE. Red plans stay clickable on purpose: pressing one shows the refusal and the failing hypotheses. Physics has veto.

Everything on screen carries a permanent REAL / SIMULATED legend.

How we built it

Two humans and four AI coding sessions (two Claude Code, two Codex) working from a frozen interface contract and a GitHub issue board, with CI that rejects any pull request touching paths outside the author's area. Thirty-six issues, one branch each, every PR reviewed against its acceptance criteria and a clean-clone test run before merge.

  • Orbit: Skyfield + SGP4 on cached elements; eclipse as the overlapped area of the apparent Sun and Earth discs, so penumbra is a fraction, not a flag. The textbook check passes: a 500 km circular orbit at zero beta sits in shadow for $35.8 \pm 1$ min.
  • Power: a truth simulator with a battery whose open-circuit voltage, resistance, efficiency and fade rate are fitted from NASA's PCoE cell-aging data; seeded faults, sensor noise and bias; a vectorised planning model for the verifier.
  • Verifier: $N = 100$ Monte Carlo samples per plan and hypothesis over array efficiency, capacity, load power, initial state of charge and sensor bias, seeded by the plan hash so the same plan always gives the same verdict. A plan passes a hypothesis when at least 95 % of samples keep $\mathrm{SOC} \ge 0.30$, bus voltage and temperature inside limits, and essential loads on.
  • Scheduler: CP-SAT with optional intervals and per-station no-overlap, greedy fallback; 200 uplink requests over 24 satellites solve optimally in about 0.3 s.
  • Agent: Gemini with function calling; a narration rule forbids the model from writing any digit, so every number on screen comes from the backend through placeholders. Recorded flight-recorder runs replay offline to the same plan hashes.
  • Frontend: CesiumJS globe that boots in under a second with no network, bundled imagery and fonts; the storm demo plays in 90 seconds with one human click.

Challenges we ran into

  • Contacts that are too short to use. Real passes can graze the 10° horizon for 30 seconds. We kept the spec (every contact is 2–15 min) by dropping unusable passes rather than loosening the test.
  • Gemini quota. The free tier gives five requests a minute and one run needs seven to thirteen. We honour the server's retry delay, cap the wait, fall back to replay, and keep every demo run recorded.
  • The model writing numbers. The first live narration was rejected three times for things like {AL-1.sat_id} and bare {H2}. The renderer now unwraps bare ids and lists the usable fields; the final step of a run can only summarise, never call tools.
  • Scheduler priorities inverted. Triage rank 1 means most urgent; the solver maximised weight, so the most urgent satellite had the lowest priority. Caught by a fleet-mode test, fixed with $w = 1/\mathrm{rank}$.
  • Agent quota during the night. Two sessions ran out; work was handed over on the board with explicit takeover notes, and the remaining sessions finished it under the original branch names.

Accomplishments that we're proud of

  • A single-satellite scenario runs end to end on the real orbit, live with Gemini and in offline replay, to the same plan hash, and the truth simulator keeps the battery above the floor through the next eclipse.
  • Measured, not asserted: diagnosis accuracy rises from 77.5 % to 90 % with peer comparison and the diagnostic test ($n = 40$). A verified plan keeps about 26× more payload time than "safe mode immediately" at the same safety ($n = 10$ storms, 42 fixes). The planning model evaluates $10{,}000$ satellites $\times$ 6 hypotheses $\times$ 3 plans $\times$ 20 samples in 29 s on a laptop.
  • One honest miss: in one of 42 cases the verifier passed a plan that the truth simulator later broke. We report it as measured; it is the most credible line on this page.

What we learned

A frozen contract and an append-only decision board let five workers (human and AI) ship 36 issues in a day without stepping on each other. The hard part of "AI for operations" is not the model; it is the envelope around it: allowlists, hashes, a verifier that can say no, and a UI that never computes a number the backend did not produce.

What's next for Penumbra

Close the verifier miss with a tighter capacity prior; conjunction avoidance coupled with power; real operator-in-the-loop timing studies instead of a fixed 20 minutes per decision.

Real vs. simulated data

Real: satellite orbits (public CelesTrak elements for one constellation, renamed SAT-01..SAT-24), Sun and Earth geometry (JPL DE421), ground-station sites (approximate coordinates), 10° contact windows, and the battery parameters fitted from NASA PCoE Li-ion aging data. BIRDS CubeSat on-orbit telemetry is used only as a sanity check of our panel-fault signature. Simulated: all power telemetry, the faults, sensor noise and bias, and the recovery plans' effects. No real operator or satellite names appear anywhere.

What existed before the hackathon

Planning documents, a draft interface contract, the issue backlog, and workflow tooling (board reader, path guard, CI skeleton). All product code was written during the hacking period. The full list is in docs/PRE-EVENT.md.

Sense → Understand → Act (Radial track)

Sense: real orbit and contact geometry, contact-limited telemetry. Understand: detector, hypothesis scoring, peer comparison, diagnostic test. Act: verified plan, scheduled uplink, flight-rule or human approval, SAFE_MODE when nothing is safe.

MLH tracks

Gemini. gemini-3.5-flash-lite with function calling drives the agent through a 12-tool allowlist; every live run is recorded and replays offline; the narration rule means the model never emits a digit. ElevenLabs. [TODO: keep only if T28 lands] Voice callouts read the agent's rendered narration at alert, approval and reveal. Vultr. [TODO: keep only if T32/T33 land] The replay demo is served from a Vultr compute instance, and the 10,000-satellite scale benchmark was re-run there.

Built With

Share this project:

Updates

Submission history