BriefLoop

Enterprise AI doesn't have a writing problem—it has an approval problem. BriefLoop is a loop that adds source-linked evidence, quality gates, and human approval—so unresolved issues can't ship.

Comment

BriefLoop

Open-source Loop Engineering for auditable business briefings. 019f504e-22f6-7bd0-be6f-2f382d5a888c

Inspiration

I work in investor relations and strategic research at a Nasdaq-listed company. One of my long-term responsibilities is preparing industry weekly reports and business memos for management. AI can produce a professional-looking draft very fast now, that part is basically solved. The hard part is something else: when the evidence is very fresh, incomplete, or contradicts itself, figuring out what actually happened recently.

During a Build Week experiment I gave GPT-5.6 a frozen package of public sources about the AI industry. The package had evidence that a product was already usable on July 8, and also a more detailed announcement published on July 16. The baseline output still wrote July 16 as the first release date. The model had found the earlier evidence, so this was not a retrieval failure. It read the material correctly and then assembled the timeline wrong.

This happens all the time in corporate research. A press release date is not the event date. A feature may sit in preview for weeks before the official announcement. An incident moves through investigating, monitoring, resolved, while old articles keep circulating. My physics training makes me want to write the problem down formally, and it is really just this:

$$t_{\text{publish}}(s) \neq t_{\text{event}}(s) \neq t_{\text{adjudicated}}(s)$$

Every source carries several different clocks, and giving the model more context does not automatically tell it which state superseded which one. Citations alone don't answer the questions that matter in a company either: did the model consider the conflicting source, which version did it use, who accepted this wording, and when an error is found later, where is the original claim, the finding, the fix and the approval record. A citation existing is not the same as evidence being considered, considered evidence is not the same as a supported claim, and a supported claim is not the same as a resolved conflict:

$$\text{cited}(c) \not\Rightarrow \text{considered}(c) \not\Rightarrow \text{supported}(c) \not\Rightarrow \text{resolved}(c)$$

OpenAI and Thrive Holdings’ Tax AI later provided a useful production precedent. It showed how practitioner corrections can become structured traces, evaluation targets, and bounded Codex engineering tasks while domain experts and engineers retain final authority. BriefLoop applies a related loop to unstructured enterprise claims, conflicting sources, and briefing release decisions

So my conclusion was that enterprises don't only need a model that writes better reports. They need an AI harness with something like internal controls, where sources, claims, versions, authority and approvals all have clear owners and persistent records. BriefLoop is my answer to that.

What it does

BriefLoop sits between the AI-generated content and the final reader who might rely on it, and it keeps the evidence and control history behind a periodic business brief. It runs the production loop and defines a second improvement loop that is not store-native yet:

production loop:
sources → claims → draft → audit → human approval → delivery

improvement loop:
production signals → structured finding → scoped fix
→ same-evidence regression → human review → release decision
→ approved guidance for the next run

The first loop governs how a single report earns the right to be delivered. The planned second loop governs how a real failure becomes a tested and approved system improvement instead of a forgotten chat message.

The permission structure has three layers. The AI proposes, Python records and enforces deterministic rules, and the human judges and approves. The model can look for sources, propose candidate claims, write analysis, flag conflicts and suggest revisions. It cannot assign authoritative claim identity by itself, cannot silently modify frozen state, cannot pass its own gates, cannot approve delivery. Python manages the schemas, hashes, transaction receipts, claim and artifact versions, workflow state transitions, deterministic gates and delivery records. Semantic disagreements that can't be safely resolved by pre-declared mechanical rules are preserved and escalated to a human instead of being smoothed over.

The current stable version is v0.14.0. In the experimental Codex path, a SQLite ControlStore is the single runtime authority, and JSON, Markdown, HTML and status pages are just replaceable projections, not independent sources of workflow truth. The store-backed production loop already has receipts, versioning, gates, approvals and delivery. v0.14.0 also includes the experimental briefloop init --web wizard and the read-only three-page briefloop quality html export. The full store-native feedback write-back is not reopened yet, and when there is no authoritative improvement data, the experimental review surface shows unavailable rather than fabricating a closed improvement loop.

How I built it

I am the only human maintainer. I handle product architecture and final merge decisions. My background is physics, finance, investor relations and strategic research, not traditional software engineering, so a lot of the engineering vocabulary in this project I learned while building it.

Codex is the main engineering environment. For example, https://github.com/Stahl-G/briefloop/pull/540 , I run different Codex sessions for architecture decomposition, Python implementation, tests, adversarial review, failure analysis, documentation and bounded fixes. My job is to define the product goals, the evidence boundaries, the system invariants, and the acceptance criteria a change has to meet before it can be merged.

GPT-5.6 did several different kinds of work in the project. In the weekly report experiment it generated candidate claims, the report drafts, and the outputs for the Prompt and Skill baselines. For the experiment design and the technical report I used GPT-5.6 Sol in Codex at the highest available reasoning effort, together with an Academic Research Skill, to find and organize related research, propose candidate experiment designs, and turn the final chosen design into a versioned experiment protocol and report drafts. After that I opened a separate ChatGPT session with GPT-5.6 Sol Pro whose only job was to attack the research questions, the control conditions, the falsifiability criteria, the citation basis and the claim boundaries.

Model output was always a proposal. I chose the research questions, verified the primary sources, fixed the experiment scope, froze the protocol, accepted or rejected revisions and approved the final release. So the research process itself followed the same division of authority that BriefLoop argues for. The Academic Research Skill organized the research workflow, but it was never treated as evidence; the real sources remained papers, official docs, repository artifacts and first-party release material.

During Build Week the project also went through an important control-plane migration. The early runtime leaned heavily on JSON files and filesystem projections. The new path uses strict Pydantic requests, a SQLite ControlStore, Unit of Work, transaction receipts and immutable artifact versions. When the first activation plan exposed serious architecture and validation problems, I stopped that path instead of patching on top of it. The recut first collected the full failure list on macOS and Windows, then separated real production defects from outdated acceptance assumptions and explicitly unsupported behavior, and only then allowed fixes. v0.14.0 makes SQLite the sole authority for new Codex runs, deletes the legacy JSON runtime-state stack and its consumer layer, and keeps retired public commands fail-closed. The public GitHub history keeps the failed first plan, the review findings, the alternative designs, the fix commits and the final merge, instead of rewriting the migration as if it succeeded from day one.

The boundary between "the model wrote something" and "the system accepted it" is enforced by an actual transaction, not by instructions. The claim freeze looks like this:

# Store-derived role action starts one recorded claim-ledger invocation
briefloop runtime invocation-start --workspace <workspace>
# claim role writes only the allowed claim draft proposal in invocation scratch,
# WITHOUT claim_id; a preassigned ID at any nesting level is rejected
briefloop runtime invocation-accept --workspace <workspace> \
  --envelope <scratch/.../role_task_envelope.json>
# the next Store action is deterministic claim freeze
briefloop runtime next --workspace <workspace> > <claim-freeze-action.json>
briefloop runtime apply --workspace <workspace> \
  --action <claim-freeze-action.json>
# Python assigns CL-#### deterministically and records the frozen ledger,
# hash, event and transaction Receipt; no state file is edited directly

After freezing, analyst and auditor roles can only read the ledger. Any change needs a new run or an explicit contamination, supersession and repair record. There is a longer historical architecture reference (v0.6.1), bound to the v0.12.1 code snapshot, in the repo：https://briefloop.ai/reports/briefloop-architecture-reference-v0.6.1.en.html

Challenges we ran into

An early reference run taught me that a written workflow is not enough. The agent produced most of the expected content artifacts but skipped the control transactions and quality gates entirely. It had understood the Orchestrator contract as background reading, not as an API it must execute. That failure became a design principle: an obligation written in a prompt is not an execution guarantee. Now agents can propose work content, but only deterministic services can decide whether an operation is allowed to change authoritative state.

Another hard problem was partial failure around transactions. A database transaction can be committed while the caller never receives the full result. For example, the system has recorded an agent invocation, and then the process dies before writing out a valid role envelope. A naive retry might cause the business effect to happen twice, but refusing the retry might leave the whole run stuck forever. The recovery path has to find the original receipt, prove the retry request is identical to the original one, reconstruct which action was legal at that time, and only fill in the missing projections without creating a second invocation. In other words, applying a request has to be idempotent:

$$f(x) = f(f(x))$$

That one line took days of failure analysis to make true on both macOS and Windows.

The SQLite migration also forced me to re-examine what many old tests were actually protecting. Some tests used to treat "this JSON file exists" as success. What the new architecture really needs to verify is whether SQLite is the only authority, whether a failed request keeps zero business writes, whether an identical request is replayed exactly instead of executed again, whether retired commands fail closed, and whether every visible output corresponds to a legitimate delivery record. The hardest part was never generating the happy path. It was deciding, over and over, what state the system should be left in after an interrupted process, a partial failure, a historical replay, or an unsupported platform.

Accomplishments that I'm proud of

Honestly, most of what I'm proud of is boring control-plane engineering, but that is exactly the point of this project. There is now a receipt-backed control path where agent output is only a proposal until a deterministic service accepts it. Claims and artifacts have explicit identity and versioning, and frozen records can't be silently rewritten. Workflow states, quality gates, approvals and deliveries are recorded separately, so a file existing no longer counts as success. The experimental Codex path runs on the SQLite ControlStore as the single authority, and there is a read-only review page that clearly separates deterministic quality projections, advisory model findings, and improvement data that is unavailable.

I'm also proud that the repo keeps the failed plans, the review findings, the rejected designs, the fixes and the exact-head validation in public, not only the final working version. And for the controlled comparison of Prompt, Skill and BriefLoop, I froze the inputs and output boundaries from the same source package, and results are only reported after outputs, hashes and blind review are all frozen.

I want to be clear about what these accomplishments prove. They demonstrate the control plane and the engineering process. They do not prove that every generated report is correct, and they do not prove yet that BriefLoop improves output quality. That claim has to wait for the frozen evaluation.

What I learned

The biggest lesson is that a prompt is not a control plane. A model can understand a rule, repeat it back to you perfectly, and still violate it when generating or executing. Any rule that actually matters needs an authoritative owner, a verification path, a failure state and a persistent record.

The second lesson is that retrieval is not adjudication. A model can retrieve the old fact and the new fact at the same time, cite both, and still assemble a wrong timeline. More context does not automatically decide source supersession, event state, or whether a disagreement should be resolved or escalated to a human. Where a conflict can be decided mechanically, the eligibility has to be declared in advance, not improvised by the model:

same normalized claim scope
AND explicit version or supersession marker
AND a valid total ordering
AND a current-value question
AND no source-authority conflict
AND no disclosure-policy conflict
→ eligible for deterministic resolution

otherwise → stays unresolved, escalated to a human

The third lesson is that the hard part of enterprise AI is both an intelligence problem and an authority problem. The questions that matter are not only what the model wrote, but also which evidence was frozen, which version the run used, who is allowed to change state, which gate passed, which conflict is still open, and who approved the delivery. Physics made me look for invariants, and years in finance and IR taught me that wording has real consequences. So BriefLoop ended up built around two constraints. First, some states must never occur — no run reaches a reader without a gate record and a named human approver:

$$\forall r:\ \text{Delivered}(r) \Rightarrow \exists\, (g, h):\ \text{GatePassed}(g, r) \wedge \text{HumanApproved}(h, r)$$

Second, the strength of the wording may never exceed the strength of the evidence behind it, which anyone who has written disclosure-adjacent text will recognize immediately:

$$w(c) \le s(c)$$

What's next for BriefLoop

The first task is finishing the current A0–A2 comparison: freeze the outputs and hashes for the Prompt, Skill and BriefLoop conditions, complete the blinded review, and publish the real observations, even if they are unfavorable to BriefLoop. The second task is running BriefLoop on a real, continuously produced IR or strategic research brief. The test is not only whether a report can be generated, but also whether another reviewer, without reading the source code, can understand the sources, claims, conflicts, revisions and approval trail. The product also needs a store-native human disposition and feedback write-back path, where only classified and human-approved review decisions can become future guidance, and which stays strictly isolated from the source evidence of the current run.

After that, the next research question is not how to add more agents, but where parallel model work actually produces measurable marginal value. My current hypothesis is that multi-agent helps most in two places: parallel search bandwidth on the evidence discovery side, and independent fresh-context review after the final draft is frozen. Claim authority, writing, gates, approval and delivery stay single-threaded.

Once A0–A2 closes, we will separately pre-register an A3 study prototyping an optional Evidence Fan-out topology: deterministic services freeze bounded research shards, multiple research workers search first-party sources, regulatory policy, competitors and counter-evidence in parallel and can only submit invocation-scoped evidence fragments, then a deterministic fan-in checks source bytes and hashes, configured date windows, exact duplicates, and whether every scheduled shard returned a result or an explicit failure record. A single claim curator only proposes the merged claim set, only Python assigns identity and freezes the canonical claim ledger, one composer writes the report, and an auditor in an independent context reads only frozen artifacts and can submit findings but never approve delivery. What runs in parallel is the model work, not the authoritative writes; SQLite transactions and artifact freezing stay serial, deterministic and replayable.

For fixed-period briefs with a known source scope, single session remains the conservative default with lower coordination complexity, and Evidence Fan-out is an explicit user opt-in for open evidence spaces where coverage matters more than cost. Most importantly, it is a falsifiable hypothesis, not a verified capability. It will be compared against the single-agent and single-session baselines on the same frozen source package, with pre-registration, blinding and repeated runs, on coverage of a pre-registered human-adjudicated material-issue set, unsupported material claims, trace completeness, wall-clock latency, total tokens and estimated cost, and variance across runs. It only earns the right to become a stable topology if it actually moves the frontier, meaning at least one of quality, latency or cost improves and none of them gets worse:

$$(Q', L', C')\ \text{qualifies} \iff Q' \ge Q\ \wedge\ L' \le L\ \wedge\ C' \le C,\ \text{at least one strict}$$

Buying results with more tokens does not count. We will publish the real results, even if they are unfavorable to BriefLoop.

Built With

  • ai-governance
  • chatgpt5.6pro
  • codex
  • enterprise-ai
  • gpt5.6sol
  • harness-engineering
  • html
  • loop-engineering
  • mit-licensed
  • open-source
  • pydantic
  • python
  • pyyaml
  • sqlite
Share this project:

Updates