CostEngine Decision Integrity Agent

Inspiration

Fleet replacement decisions can move millions of dollars, yet a polished result can still rest on the wrong comparator or incomplete evidence. I spent nine years as a principal investigator at the National Renewable Energy Laboratory building vehicle energy models. CostEngine is that physics, productized: an evidence-governed system for fleet decisions that must be explainable, auditable, and defensible.

The Build Week extension began with a real failure mode: comparing an electric bus against a mismatched acquisition baseline. We wanted an expert correction to do more than repair one result. It should become durable regression evidence that prevents the same defect from returning.

What it does

CostEngine Decision Integrity Agent lets a user challenge the comparator or evidence basis behind a fleet replacement decision.

The agent can invoke exactly two read-only tools:

• audit_comparator_integrity checks comparator coherence and same-baseline gates.
• audit_evidence_integrity checks evidence completeness and public provenance labels.

A deliberately malformed generic 40-foot transit-bus fixture—continuously labeled historical defect reproduction—is blocked by deterministic gates I16 and I31. The corrected same-baseline fixture passes the identical challenge.

The workflow then creates a proposed regression seal. It remains PROPOSED_REQUIRES_EXPERT_APPROVAL; no model can approve it or add it to the permanent test suite.

How we built it

The Build Week extension is isolated to CostEngine’s web layer. A feature-flagged Python orchestration module implements a constrained Responses API tool contract. The browser sends only the user’s challenge and a reference to a server-held fixture; it cannot supply or alter economic inputs.

CostEngine’s existing deterministic engine remains authoritative for all numbers, formulas, comparator pairs, validation gates, and procurement conclusions. Model-generated prose is rendered separately from deterministic results.

The implementation fails closed when it encounters malformed payloads, unexpected tools, runtime failures, invalid mappings, or inconsistent aggregate results.

How we used GPT-5.6 and Codex

GPT-5.6 in Codex helped translate an owner-approved safety boundary into the orchestration module, served user interface, payload-scrubbing controls, adversarial tests, historical-label guard, and regression verification.

The implemented runtime contract permits GPT-5.6 to interpret a user’s challenge, invoke only two allowlisted read-only integrity tools, and explain sanitized public results. It cannot select authoritative inputs, originate or alter an economic value, override a validation gate, repair a failed comparison, or approve a regression seal.

CostEngine remains a deterministic, evidence-governed decision system. GPT-5.6 serves as a constrained orchestration and explanation layer: it interprets a user's challenge and invokes two read-only integrity tools. It does not originate, calculate, or alter fleet economics or procurement conclusions. All displayed numbers and gate outcomes come from CostEngine's deterministic engine and validation gates. Codex was used to implement and test this Build Week extension. Any proposed regression seal requires expert approval before it enters the permanent test suite.

Challenges we faced

The hardest problem was creating a strong agentic workflow without giving the model authority over high-consequence economic calculations.

We also had to ensure that an intentionally malformed historical fixture could never be mistaken for current production output. The UI therefore keeps the historical defect reproduction label visible during selection, loading, results, and transitions.

A separate challenge was preserving IP and provenance boundaries. Four standard heavy-duty certification-cycle encodings are not redistributed while their public-source archive remains pending. Their hashes, negative-control behavior, and clean-room CARB/EPA remediation path are documented in the reviewer repository.

Accomplishments

• Historical mixed-baseline case is deterministically blocked by I16 and I31.
• Corrected same-baseline case returns PASS using the same challenge.
• Regression evidence remains proposed until expert approval.
• The focused BW-1 and adjacent regression set passed 78 tests.
• With the four provenance-gated standard-cycle inputs mounted locally and hash-verified, the canonical seal suite passed 47/47 before and after the extension.
• The existing CostEngine engine source tree remained unchanged.
• The private reviewer repository contains a clean three-commit history separating the pre-BW-1 snapshot, implementation, and verification evidence.

What we learned

The strongest role for a model in high-consequence analytics is not to manufacture an answer. It is to make a user’s challenge operational under a narrow tool contract while deterministic systems retain authority over evidence, calculations, validation gates, and approvals.

What’s next

Approved correction patterns can enter the permanent regression suite after expert review. We will also rebuild the four standard-cycle fixtures clean-room from authoritative CARB/EPA sources, preserve source and transformation provenance, and intentionally re-anchor the affected baselines.

Future versions can extend the same governed challenge contract to additional fleet decision artifacts without expanding model authority over economics.

Evidence status

This submission’s recorded demonstration uses deterministic protocol simulation and does not include a live GPT-5.6 call. No simulated result is represented as a live model response.

Built With

• analytics
• api
• codex
• css
• deterministic
• fleet
• gpt-5.6
• html
• javascript
• openai
• python
• responses