Murmuration

AI agents can agree for the wrong reason. Murmuration traces whether their evidence is truly independent, preserves critical dissent, runs the missing test, and leaves the final decision with a human.

Comment

Inspiration

I am a digital archivist, not a software engineer.

My work is built around provenance: preserving where information came from, how it changed, who handled it, how different records relate to one another, and what makes something trustworthy.

In archives, a record can look convincing while still being misleading if its origin or context has been lost. Authenticity does not come from the record alone. It comes from the evidence around it: its source, history, relationships and chain of custody.

While thinking about multi-agent AI systems, I realised they have a similar problem.

Several AI agents may agree, but that agreement does not necessarily represent several independent reasons. They may all be repeating the same source, relying on the same hidden assumption, or inheriting the same omission. A synthesiser may make the result sound more confident without introducing any genuinely new evidence.

That led to the central question behind Murmuration:

What if AI-assisted decisions were treated like accountable records, with provenance, evidence lineage, preserved dissent, verification and human authorisation?

Murmuration applies archival provenance principles to multi-agent decision-making. It preserves the chain from source evidence, to agent interpretation, to claims, collective judgement, executed verification, belief revision and final human action.

I conceived, designed and built the prototype using Codex as my primary technical collaborator. I brought the domain insight, product vision, trust model and interface direction, while Codex helped me translate those ideas into a working software system.

The result brings archival thinking into AI reliability. It does not only ask what the agents decided. It preserves how they came to believe it.

What it does

Murmuration is a provenance-aware decision-review layer for multi-agent AI systems.

It sits between an AI-agent workflow and the action that workflow wants to take.

Instead of simply counting how many agents agree, Murmuration examines the structure of the evidence behind their recommendations.

It:

  • traces every recommendation back to its source evidence;
  • measures independent evidence lineages rather than counting votes alone;
  • identifies when several agents are relying on the same source;
  • preserves critical dissent instead of averaging it away;
  • records structured claims, assumptions, concerns and confidence;
  • identifies when an unresolved uncertainty can be answered through verification;
  • allows an AI Examiner to select only from predefined, allow-listed tests;
  • runs deterministic verification rather than asking a model to imagine the result;
  • returns observed evidence to the agents;
  • records how their recommendations change;
  • leaves the final authorised action with an accountable human.

The public demonstration follows a proposed business-records migration.

Three specialist AI agents review the change:

  • a Migration Reviewer;
  • an Operations Reviewer;
  • a Rollback Reviewer.

The Migration and Operations Reviewers approve. The Rollback Reviewer blocks.

A simple majority appears to support proceeding, two votes to one.

Murmuration reveals, however, that both approving agents relied on the same migration brief. Their two recommendations therefore represent only one independent supporting lineage.

The Rollback Reviewer had access to a different operational source. It raised a critical concern: every original historical record location must be recoverable exactly if the migration is rolled back.

No available evidence proved that this was possible.

Murmuration therefore marks the case TEST REQUIRED.

The GPT-5.6 Examiner selects a predefined, allow-listed rollback test. Deterministic code validates the selection and executes the test against a controlled SQLite environment.

The test applies the migration, performs the rollback and compares the restored records with the originals.

It fails.

Two manually overridden historical locations are replaced with generic reconstructed paths. The original values are not restored exactly.

That observed result is returned to the agents.

Both original approving reviewers revise their recommendations from APPROVE to BLOCK. The Rollback Reviewer remains at BLOCK, and the revised Decision Synthesizer also recommends blocking the action.

The original migration claims were not necessarily false. They were simply no longer sufficient to justify proceeding once the failed rollback evidence became available.

The final deterministic gate remains BLOCKED.

An accountable human can then:

  • confirm the block;
  • escalate the case for further investigation;
  • or explicitly override and approve with a recorded rationale.

The human disposition is recorded separately. It does not rewrite the original evidence, test result or completed agent run.

A separate safe control case demonstrates that Murmuration does not block everything. When a decision has genuinely independent support and no unresolved critical concern, it can proceed to human review without unnecessary intervention.

How I built it

I approached the project from an archivist’s perspective rather than beginning with a conventional software architecture.

I first modelled the decision as a record chain:

source evidence → agent interpretation → claim → collective decision → verification → belief revision → human-authorised action

Each stage needed to remain connected to what came before it.

From that model, I designed a system that separates semantic judgement from deterministic control.

GPT-5.6 Terra performs the semantic work

GPT-5.6 is used for:

  • specialist interpretation of permission-filtered evidence;
  • producing structured claims, assumptions, concerns and confidence;
  • synthesising validated reviewer positions;
  • identifying which unresolved question matters;
  • selecting an appropriate test from an allow-list;
  • revising recommendations after observed evidence is returned.

The project uses five AI roles:

  1. Migration Reviewer
  2. Operations Reviewer
  3. Rollback Reviewer
  4. Decision Synthesizer
  5. Examiner

The Examiner cannot invent shell commands, arbitrary SQL or new tests. It can only select a registered test identifier.

Deterministic TypeScript performs the governing work

Deterministic code is responsible for:

  • evidence identifiers and permissions;
  • validating citations;
  • preserving source lineage;
  • calculating evidence independence;
  • evaluating gate conditions;
  • enforcing the verification allow-list;
  • executing SQLite verification;
  • comparing database snapshots;
  • storing protected run and evaluation records;
  • controlling the final gate state.

This separation is central to Murmuration:

Models interpret. Deterministic code governs. Tests observe. Humans authorise.

The public interface is a React and Vite decision-review workspace served through a Node.js application.

It guides the user through five stages:

  1. Decision
  2. Evidence
  3. Verification
  4. Revision
  5. Authorise

The Technical Audit area exposes the trust boundaries, deterministic controls, safe comparison case, evaluation results, hashes, replay commands and limitations.

The hosted version is replay-only. Opening or navigating it makes no GPT, Codex SDK, Platform API or paid API call.

It displays an unchanged completed GPT-5.6 run and its real deterministic verification outputs.

How I used Codex

Codex was my primary technical build collaborator.

I used it to help translate the archival and product concepts into an implemented system.

Codex helped me:

  • explore and refine the architecture;
  • create the data schemas;
  • implement provenance and lineage logic;
  • build the deterministic gate;
  • create the SQLite verification engine;
  • structure the live multi-agent workflow;
  • build the React interface;
  • write and run automated tests;
  • protect saved run records;
  • improve accessibility and usability;
  • prepare the public repository;
  • scan for secrets and local paths;
  • create the deployment configuration;
  • deploy the replay through Render;
  • and generate the first cut of the demonstration video.

I remained responsible for the product concept, domain model, scenario, trust boundaries, user experience, evaluation and final decisions.

The development process was deliberately controlled.

I worked in small milestones, reviewed the output at each stage, challenged unclear or overly technical design decisions, and repeatedly constrained Codex so that model interpretation could not bypass deterministic permissions, test allow-lists, protected evidence or human authorisation.

Codex also ran validation after each significant change and stopped before commits, pushes, model calls or deployments when instructed.

This allowed me to build a technically credible project without beginning as a conventional programmer, while still retaining ownership and understanding of the system.

Challenges

The hardest challenge was making the difference between agreement and independent support visible.

It would have been easy to display several agent answers and call that a multi-agent system.

Murmuration needed to answer more difficult questions:

  • Did the agents see the same information?
  • Did their recommendations come from independent sources?
  • Was a minority concern preserved?
  • Which claims were supported?
  • Which assumptions remained untested?
  • What new information would actually resolve the uncertainty?
  • Did the agents revise their beliefs after observing the result?
  • Who remained accountable for the final action?

Another challenge was designing safe verification.

The Examiner needed enough flexibility to decide which uncertainty mattered, but it could not be allowed to generate arbitrary commands, access hidden fixture truth or execute unrestricted SQL.

The solution was to separate test selection from test execution. The model selects only an allow-listed test ID. Deterministic code decides whether that test is permitted and performs the execution.

The interface was also challenging.

Early versions looked more like a technical report than a usable product. I iteratively redesigned the experience around a clear human journey: understand the decision, inspect the evidence, review the test, see what changed and authorise the outcome.

Accomplishments

I am particularly proud that Murmuration:

  • exposes false confidence created by duplicated evidence;
  • distinguishes vote count from independent support;
  • preserves a minority critical concern;
  • commissions and executes a real verification test;
  • records explicit before-and-after agent positions;
  • preserves the original evidence and run records;
  • produces an inspectable decision chain;
  • demonstrates both an unsafe case and a safe control;
  • leaves the final authorised action with a human;
  • and is publicly testable without rebuilding the project.

The completed project includes:

  • a completed live GPT-5.6 Terra multi-agent run;
  • three specialist agent roles;
  • a Decision Synthesizer;
  • an Examiner;
  • deterministic provenance and gate logic;
  • registered SQLite verification;
  • protected replay records;
  • safe and unsafe scenarios;
  • four-case deterministic gate evaluation;
  • 51 automated tests;
  • a public GitLab repository;
  • a live Render deployment;
  • and a public demonstration video.

The flagship result shows:

  • 2 approving reviewers;
  • 1 independent supporting lineage;
  • 1 critical concern;
  • an initial state of TEST REQUIRED;
  • failed exact-rollback verification;
  • 2 reviewers revising from approve to block;
  • and a final deterministic gate of BLOCKED.

What I learned

I learned that multi-agent reliability is not primarily a voting problem.

It is a provenance problem.

Adding more agents does not necessarily add more knowledge. Confidence should increase only when genuinely new and independent evidence enters the decision.

I also learned that AI systems need a clear division of responsibility.

Models are useful for interpretation, comparison, synthesis and revision. They should not be treated as the source of mechanical truth when a real test can observe the result.

I learned that dissent has value.

The Rollback Reviewer was initially outnumbered, but its concern was independently grounded and testable. Preserving that concern prevented a superficially popular but unsafe action from proceeding.

On a personal level, I learned that domain expertise can be translated into working software without beginning as a conventional programmer.

The important part was having a clear model of the problem, defining the boundaries carefully, questioning weak assumptions and using Codex as a disciplined implementation partner rather than handing over the product thinking.

What's next

The current prototype demonstrates one controlled business-records migration workflow with a curated verification registry.

In a production organisation, Murmuration could integrate into an existing agent workflow like this:

source systems → evidence retrieval → specialist agents → Murmuration → registered verification → human authorisation → originating workflow

The originating organisation would decide:

  • which evidence sources each specialist may access;
  • how provenance metadata is captured;
  • which tests are registered;
  • which concerns require intervention;
  • which actions require human authorisation;
  • and what must be retained as an audit record.

Future development could include:

  • additional domain-specific verification adapters;
  • integrations with existing multi-agent orchestration platforms;
  • enterprise identity and access control;
  • configurable evidence-permission policies;
  • durable human disposition records;
  • dashboards covering multiple active decisions;
  • alerts for duplicated or circular evidence;
  • more sophisticated lineage visualisation;
  • and evaluation across software, finance, compliance, operations and public-sector workflows.

The project does not claim to prove general AI safety or replace professional judgement.

Its purpose is narrower and practical: to make AI-assisted decisions easier to inspect, test and hold accountable before they become actions.

Built With

Share this project:

Updates