FORGE - AXIOM Guard

AXIOM Guard validates Codex and GPT-5.6 engineering specifications against deterministic constraints, returns corrective feedback, and preserves an auditable retry trace before acceptance.

Comment

Inspiration

Large language models (LLMs) can generate convincing engineering specifications, code, and models, but fluent output is not the same as valid engineering.

A specification may contain values that look reasonable individually while violating a coupled physical, mathematical, or geometric constraint. In engineering, that difference matters.

FORGE began as a research platform developed for my master’s thesis on neuro-symbolic supervision of AI-generated engineering artefacts. The central idea became AXIOM: let the language model generate freely, but place deterministic validation and a controlled retry loop between generation and acceptance.

For OpenAI Build Week, I connected this architecture to GPT-5.6 and turned the research prototype into a focused, inspectable developer tool.

Battery cells are the demonstration domain because they combine electrochemical, mathematical, material, and mechanical constraints. The underlying supervision pattern is reusable wherever an AI-generated artefact can be checked against explicit deterministic rules.

What it does

AXIOM Guard supervises GPT-5.6-generated engineering specifications, demonstrated through lithium-ion battery-cell design.

The workflow is:

  1. An engineer describes the required battery cell in natural language.
  2. GPT-5.6 generates a structured engineering specification.
  3. FORGE parses the specification and checks its schema.
  4. Deterministic rules evaluate physical relationships, geometry, materials, and calculated values.
  5. If a constraint fails, AXIOM rejects the attempt and produces machine-readable corrective feedback.
  6. GPT-5.6 generates a complete corrected specification.
  7. AXIOM accepts the design only after all declared constraints pass.
  8. FORGE converts and calculates the accepted design.

The interface preserves the audit trail: the original prompt, every generated attempt, failed constraints, corrective feedback, token usage, model identity, provenance hashes, and the final decision.

Passing the declared constraints does not prove that a design is completely safe or production-ready. It provides a transparent, deterministic supervision boundary around probabilistic generation.

How we built it

FORGE uses a three-layer neuro-symbolic architecture:

  • Der Generator: GPT-5.6 creates structured engineering proposals.
  • Der Validator:: deterministic Python checks evaluate schema compliance and engineering constraints.
  • Der Supervisor: AXIOM controls retry logic, termination conditions, acceptance, rejection, and trace preservation.

The existing FORGE platform already included the battery-cell calculation engine, constraint registry, AXIOM retry loop, FastAPI endpoints, Streamlit interface, and support for local and cloud language models.

The OpenAI Build Week contribution added:

  • GPT-5.6 integration through the existing backend abstraction;
  • the OpenAI Responses API client layer;
  • mapping of token usage and returned-model metadata into the existing telemetry model;
  • offline backend, API, and pipeline tests;
  • capture of an authentic GPT-5.6 constraint-correction trace;
  • trace hashing and a downloadable audit bundle;
  • a focused live and verified-replay interface;
  • an interactive Colored Petri Net visualisation of the supervised workflow;
  • documentation separating the pre-existing research platform from the Build Week work.

Codex supported the implementation by inspecting the existing architecture, making bounded changes, writing tests, reviewing diffs, and preparing the application and documentation.

Challenges we ran into

The main challenge was not forcing GPT-5.6 to fail. That would have produced an artificial demonstration with little engineering value

The goal was to find an authentic case where GPT-5.6 produced a structurally valid and individually plausible specification that violated a coupled engineering constraint.

The selected run involved a compact cylindrical cell. Most individual parameters passed, but the combination was geometrically impossible: the calculated winding span was only 1.7 mm, below the required 2.0 mm.

AXIOM rejected the first attempt and returned precise corrective feedback. GPT-5.6 then increased the cell diameter, producing 2.2 mm of winding space. The corrected specification passed all declared constraints.

Another challenge was keeping the Build Week scope focused. FORGE contains battery calculations, CAD, simulation, machine-learning, API, and experimental infrastructure, but the demonstration needed to communicate one clear story:

Proposal → Deterministic Rejection → Corrective Feedback → Retry → Acceptance

Accomplishments that we're proud of

The underlying AXIOM architecture was developed and evaluated before Build Week across multiple cloud and local language models.

That research showed that deterministic supervision can significantly improve the validity of generated engineering specifications, and that responsiveness to corrective feedback is a distinct capability from first-attempt generation quality.

During Build Week, this research architecture became an inspectable developer-facing workflow using GPT-5.6.

The completed demonstration includes:

  • an authentic GPT-5.6 first-attempt engineering failure;
  • a schema-valid but geometrically inconsistent specification;
  • deterministic rejection through constraint CY5;
  • actionable corrective feedback;
  • a successful second GPT-5.6 attempt;
  • an accepted and calculated battery-cell design;
  • verified replay without requiring an API key;
  • live OpenAI execution for users who provide their own key;
  • a step-by-step Colored Petri Net replay;
  • cryptographically hashed trace and audit bundles.

We also maintained explicit provenance:

  • the original FORGE repository was frozen at a tagged baseline;
  • all Build Week changes were isolated on a dedicated branch and worktree;
  • genuine GPT-5.6 traces were preserved;
  • verified replay is clearly distinguished from live execution;
  • no constraint was weakened to manufacture a successful demonstration;
  • the originating commit is recorded in the trace.

What we learned

A stronger language model does not remove the need for deterministic validation.

The useful division of responsibility is:

  • the model proposes;
  • deterministic tools calculate and validate;
  • the Supervisor controls the process;
  • the engineer inspects the evidence.

We also learned that corrective feedback must be precise. Telling a model only that a value is invalid is often insufficient. Effective correction identifies the violated relationship, the calculated value, the accepted threshold, and an actionable direction for revision.

The battery-cell example also showed why coupled validation matters. Individual values may all appear reasonable while their combination is physically or geometrically impossible.

What's next for FORGE

The next step is to generalise the AXIOM supervision pattern beyond the battery-cell demonstration.

Potential engineering artefacts include:

  • technical requirements;
  • system models;
  • traceability records;
  • verification methods;
  • engineering calculations;
  • CAD parameters;
  • configuration and design specifications.

Each domain would require its own schemas, deterministic rules, and validators, while reusing the same generation, supervision, retry, provenance, and audit architecture.

The broader goal is not to replace engineering judgement. It is to make AI-generated engineering work inspectable, correctable, and governable before it is accepted.

Built With

  • api
  • codex
  • colored-petri-net
  • docker
  • fastapi
  • github
  • gpt-5.6
  • json
  • neuro-symbolic-ai
  • openai
  • pydantic
  • pytest
  • python
  • pyyaml
  • rest
  • schema
  • streamlit
Share this project:

Updates