AEOLUS

AEOLUS simulates a space habitat's air system to spot faults like a broken fan early, before the air gets unsafe. Every run is repeatable, so the results can always be trusted and checked.

Comment

Inspiration

Habitat ventilation is a useful Physical AI problem because a controller cannot rely on predictions alone. Sensors can fail, actuators can behave differently from their commands, and a confident but incorrect decision can make the physical state worse.

We built AEOLUS to explore one question: how can an AI system help a simulated habitat respond to faults without becoming the final safety authority?

Arm optimization result

AEOLUS includes a controlled FP64-to-FP32 optimization of its forecast workload, measured on native Arm64 rather than emulation.

Both artifacts ran the same batch-one, 8-step x 51-target prediction workload on an Arm Neoverse-N2 runner. The FP32 artifact:

  • reduced raw model-array memory by 50% (28,759,024 -> 14,379,512 bytes)
  • reduced artifact file size by 47.1% (2,126,337 -> 1,124,273 bytes)
  • reduced median inference latency from 738.349 microseconds to 426.358 microseconds, a 1.73x speed-up
  • reduced p95 latency from 797.245 microseconds to 445.057 microseconds, a 1.79x speed-up
  • preserved prediction quality: maximum normalized drift was 4.879 x 10^-6, passing the predeclared 1 x 10^-4 parity gate

The benchmark used native aarch64, Python 3.11, NumPy 2.4.6, one CPU thread and 200 measured iterations per artifact. The artifact hashes, hardware environment, workload and results are bound into a reproducible benchmark receipt.

This is deliberately a bounded claim: AEOLUS demonstrates reduced-precision model size and CPU inference gains on native Arm64. We do not claim INT8, NPU, NEON, power, thermal, physical deployment or certification results.

The current trained 2.1-million-parameter MLP also runs natively on Arm Neoverse-N2. Its complete 8-step, 51-target forecast measured 192.7 microseconds median latency across 1,000 repetitions - approximately 5,000 forecasts per second. This is a separate native-Arm performance baseline, not the controlled FP64-to-FP32 optimization comparison.

What it does

AEOLUS is a deterministic, replayable simulation of an eight-zone habitat ventilation system.

It models airflow, pressure, fan power, electrical use, oxygen, carbon dioxide, humidity and temperature. Scenarios can include fan degradation, increased resistance, jammed dampers, biased sensors, drifting sensors and stuck sensor readings.

The simulator separates operational telemetry from hidden simulator truth. This prevents a model from cheating by reading fault labels, random seeds, future events or internal state that would not be observable in a real system.

AEOLUS also includes a deterministic Habitat Management Computer. It receives a verified plant snapshot, evaluates a proposed action and applies fixed safety rules before any command reaches the simulated plant. Learned output is advisory only and cannot bypass this authority boundary.

On top of that boundary sits a trained action-conditioned forecaster. Given the last 16 steps of telemetry and one candidate action, it predicts the next 8 steps of every zone's atmosphere. Each step, an adviser forecasts every allowed action and proposes the safest predicted future to HMC, which can accept, override or reject it. If any sensor evidence is missing, the adviser abstains and HMC runs alone. The model never commands.

Alongside the forecaster, AEOLUS includes a second, independently governed learned lane: the Issue #56 action-risk line. A frozen deterministic risk baseline (V3) was established first through a paired controller race across 32 habitat families in 16 matched condition groups. A challenger model family (V4) was then trained on a replayable counterfactual corpus - 1,664 labeled samples and 1,696 strictly replay-verified traces - and evaluated under protocols preregistered before each study ran. In its final form, the V4 advisory model outperforms the frozen V3 baseline in all six held-out evaluation families while HMC still accepts, modifies, or rejects every proposal. The model never commands.

Every run produces a SHA-256-linked control trace. Validation goes beyond checking hashes. It rebuilds the decisions and physical transitions from the original scenario, rejecting incomplete, reordered or physically inconsistent evidence even if every hash has been recalculated.

How we built it

AEOLUS is written in Python using NumPy, versioned JSON contracts and a test-driven workflow.

We built the system in bounded layers:

  1. environmental conservation and accounting
  2. explicit operating modes
  3. a multi-zone airflow network
  4. physical and sensor fault injection
  5. commanded versus achieved actuator feedback
  6. operational observability qualification
  7. deterministic HMC arbitration
  8. complete control-trace validation and replay
  9. a learned forecaster trained on 23,400 archived simulation examples
  10. a pre-registered paired closed-loop evaluation: 238 runs, each scenario executed twice with identical noise and seeds, once with HMC alone and once with the model advising
  11. a frozen action-risk baseline race: 32 paired habitat families, deterministic controller arms, and a byte-frozen V3 risk baseline whose refit must reproduce a recorded artifact hash
  12. a replayable counterfactual V4 corpus: per-action, per-horizon safety labels with hidden simulator truth excluded, checked by an independent strict-replay validator before any model touched it
  13. a preregistered V4 model-study line: ten protocol revisions, each committed before its study ran, with every negative result published unchanged

The forecaster is a compact multilayer network trained with held-out scenario clusters, so its score is measured on habitats it never saw during training. At inference it runs as pure NumPy from a hash-pinned artifact, so anyone can reproduce our numbers with one command.

Challenges we ran into

The hardest problem was distinguishing evidence integrity from evidence truth.

A trace can have perfectly valid hashes and still tell a false story if someone changes a decision and recalculates the chain. An earlier validator accepted this kind of internally consistent forgery. We replaced it with one authoritative validation route that replays both the deterministic HMC policy and the plant behaviour.

We also had to prevent hidden simulator information from leaking into operational telemetry. Many useful values exist inside the simulation, but a real deployed model would not observe them. Removing those shortcuts made the evaluation harder but more honest.

Another challenge was resisting the temptation to force an AI success. A learned component must outperform a strong deterministic baseline on untouched scenarios. If it does not, AEOLUS records that result rather than weakening the baseline. The Issue #56 action-risk line tested this rule harder than we expected: seven consecutive preregistered protocols failed to beat the frozen baseline. Instead of tuning after the fact, we diagnosed why - and proved two structural impossibilities rather than model failures. The evaluation population contained exactly one family group that ever produced risk events, and the frozen baseline already took every genuinely useful intervention there; meanwhile any split that made the evaluation decidable starved the challenger of eventful training data. We published both findings unchanged, revised the simulation fixture under explicit authorization so every condition group produces events, re-preregistered from scratch, and only then did the challenger win - on terms fixed before the run.

Evaluating the forecaster raised a subtler version of the same problem: a model can look accurate on average and still be useless for control. We therefore scored it on physical outcomes in closed loop - threshold exceedances prevented - with the scoring rules frozen before any results were seen, rather than on forecast accuracy alone.

Accomplishments that we're proud of

  • The current merged candidate passes 1,207 automated tests.
  • Clean wheel installation, deterministic replay, linting and compilation all passed.
  • The recovery evaluation covered 252 scenario families and 1,008 traces.
  • Protection activated in all 79 harmful physical-airflow families.
  • Healthy activations, wrong-zone actions, repeated protection episodes and invariant violations were all zero.
  • Median integrated physical CO2-excess reduction was 80.396%.
  • Observability detected concern in all six harmful qualification fixtures with no false concern in the healthy fixture.
  • The system correctly abstained from exact diagnosis in a deliberately ambiguous case.
  • Adversarial tests proved that recalculating every hash cannot make a policy-impossible control decision valid.
  • The forecaster scored 0.1146 normalized error on 17 untouched scenario clusters, versus 0.2880 without action conditioning - knowing the action cuts forecast error by 60%.
  • Across 102 paired fault scenarios, the model-advised controller was safer in 78, equal in 24 and worse in none; 72 advised runs finished with zero safety-threshold exceedance.
  • In the recorded demo scenario, canonical HMC drifted past the CO2 warning line for 29 steps (integrated exceedance 19.94); the advised arm acted before the first violation and never crossed it (0.0).
  • HMC overrode 81 of 793 model proposals during the campaign - the authority boundary was exercised, not bypassed.
  • Every result replays bit-for-bit from a fresh clone: identical numbers, identical trace hashes.
  • The Issue #56 V4 advisory model outperforms the frozen deterministic V3 baseline in all six evaluation families - 6 wins, 0 ties, 0 losses - with per-family safety-exposure reductions of 33.3% to 61.9%, six admitted interventions against the baseline's two, aggregate paired safety exposure strictly better (-3.42e-04), zero HMC mismatches and zero emergency overrides, under a protocol committed before the study ran.
  • Seven consecutive negative preregistered results were published unchanged, including two proven structural impossibilities - the win came from fixing the evaluation fixture and split, not from weakening the gates or the baseline. A rollback tag preserves the prior, more conservative winning configuration.

What we learned

Repeatable simulation is not the same as physical validity, and a green test suite is not the same as safe control.

We learned to keep model advice separate from actuator authority, operational telemetry separate from evaluator-only truth, and cryptographic integrity separate from semantic validity.

We also learned that detecting a problem, locating the affected subsystem and naming the exact fault are different claims. A trustworthy system should abstain when the available evidence does not support an exact answer.

The forecaster taught us the same lesson one level up: an advisory model can change physical outcomes measurably - but the safety margin is bought with consumables (advised runs used a median of 757 Wh more battery), and a model trained on complete telemetry must refuse to guess when sensors are missing. Honest trade-offs and honest abstention are features, not admissions.

Preregistration turned out to be a discovery tool, not just a discipline. Because every gate was fixed before the runs, the negative results were informative enough to prove that the evaluation fixture - not the model - was the binding constraint, and that proof is what made the eventual win credible.

What's next for AEOLUS

The learned-advisory question is now answered at development-evidence level on two independent lines: the forecaster campaign showed advising improves closed-loop safety outcomes (78/24/0), and the Issue #56 action-risk line concluded with the V4 model outperforming the frozen deterministic baseline on every evaluation family under preregistered per-family criteria. Both remain simulator development evidence; the model is advisory-only and HMC retains sole command authority. The next tier is an availability-aware model that can still forecast when sensors fail — our current model verifiably abstains instead — trained on a new corpus with explicit missing-data evidence.

Built With

Share this project:

Updates

Submission history