Tacit

• 

• 

• 

• 

Track

Track 1 — Automated Agent Engineering

Tacit is a self-improving agent that learns operational judgment from human corrections, earns autonomy only where evidence supports it, and weakens or supersedes what it learned when reality changes.

Inspiration

Every team has rules that aren't really written anywhere. They're learned through work.

A payment arrives slightly short. Someone experienced recognizes the pattern, fixes it, and moves on. When the same situation appears again, another person often has to make essentially the same judgment.

That made us think about a broader problem with AI agents.

Every time a human corrects an agent, they're teaching it something. But most agents either forget that correction or store it as context to retrieve later.

We wanted to explore a harder question:

Can an agent learn the judgment behind a correction, while also learning where that judgment stops being safe?

That became Tacit.

We chose finance as the proving ground because this couldn't be a toy. If an agent overgeneralizes something it learned while reconciling money, the mistake has consequences.

What it does

Tacit learns unwritten operational rules from human corrections and turns them into structured, evidence-backed policies.

For this project, we focused on cash application and bank-to-invoice exceptions.

On the first accounting close, Tacit encounters situations it doesn't know how to handle and sends them to a human reviewer.

The reviewer might explain:

"These invoices were paid together. The difference is an intermediary bank fee."

Tacit turns that correction into structured knowledge about the situation, action, context, and supporting evidence.

When the underlying situation appears again, Tacit can reuse that learned judgment instead of asking the human or model to solve the same problem again.

But similarity alone isn't enough.

A policy learned from an Acme international wire should not suddenly govern BluePeak. It shouldn't even govern an Acme payment arriving through a different payment rail if the evidence doesn't support that transfer.

Tacit checks the conditions under which a policy was learned before allowing it to act.

Learned policies aren't permanent either.

As evidence accumulates or contradicts previous behavior, policies move through a lifecycle:

Candidate → Active → Weakened → Superseded

So Tacit doesn't just learn what worked.

It learns where that knowledge applies and when it should stop believing it.

How we built it

Tacit deliberately separates learning from authority.

Reviewer corrections are interpreted into candidate policies using GLM-4.7-Flash through TensorMux.

The model can propose structured judgment, but it cannot set confidence, activate a rule, broaden its scope, change its lifecycle state, or authorize an autonomous resolution.

Those decisions belong to Tacit's deterministic evidence engine.

Every model proposal is validated against the actual case and its source references before it can enter the learning system. If inference returns malformed output, invents unsupported evidence, omits required provenance, times out, or becomes unavailable, Tacit fails closed and falls back safely.

This separation also gives Tacit an important efficiency property.

Inference is useful when Tacit needs to interpret new judgment. Once that judgment has been grounded and learned, future matching situations can be evaluated deterministically rather than repeatedly asking a model to rediscover the same answer.

We also used Agent Orchestrator (AO) as part of the engineering process around Tacit.

Rather than using AO only to generate code, we used it to adversarially review the learning architecture. An AO orchestrator coordinated dedicated red-team workers examining contextual scope, policy drift, model authority, provenance, ambiguity, and inference failures, along with independent reviewers checking their findings.

That gave us two learning loops:

Inside Tacit: human correction → learned judgment → application → contradiction → adaptation.

Around Tacit: implementation → AO adversarial review → discovered weakness → fix → regression testing.

Challenges we ran into

The hardest problem wasn't getting Tacit to remember.

It was preventing Tacit from learning too much from one correction.

Two situations can look almost identical while differing in one condition that completely changes whether a learned policy is safe to apply.

For example, an international-wire fee pattern learned for one customer shouldn't automatically transfer to another customer. Even for the same customer, a domestic ACH that looks similar may fall outside the evidence boundary of the original rule.

This led us to make contextual scope a first-class part of learned judgment, including conditions such as customer, payment rail, currency, variance, timing, and other supported evidence.

Model reliability created another challenge.

During live TensorMux testing, GLM-4.7-Flash returned a semantically useful candidate policy but omitted a required bank-line source reference. Tacit rejected the proposal rather than weakening its provenance requirements.

A later live inference request timed out entirely. Tacit again failed closed and retained deterministic behavior.

Then AO found something our existing tests had missed.

A model candidate could include the required case and bank-line references while also adding an invented source reference. An invoice-dependent candidate could also omit relevant invoice provenance.

The flaw could not directly authorize an autonomous resolution, but it could allow misleading provenance to enter memory.

We tightened the grounding layer so every source reference must exist in the current case and invoice-dependent policies must cite their relevant invoices. We added regression coverage, had the fix independently reviewed, and reran the complete evaluation suite without changing engine semantics.

That experience reinforced one of Tacit's core principles:

The model can suggest. Evidence decides what earns the right to be remembered.

Accomplishments that we're proud of

We're proud that Tacit became more than a demo of an agent "remembering" corrections.

We built a complete learning lifecycle that can demonstrate three different forms of reasoning:

I know this.

Tacit recognizes a previously learned situation and safely reuses the judgment.

This looks similar, but I don't know this.

Tacit recognizes that a familiar-looking situation falls outside the evidence boundary and escalates it instead of generalizing.

I used to know this, but reality changed.

Tacit detects contradictions, weakens stale knowledge, and can eventually supersede it with a better-supported policy.

Across three simulated accounting closes and 81 exception cases:

Human reviews: 27 → 8 → 11

Autonomous resolutions: 0 → 19 → 16

Incorrect autonomous applications: 0

Tacit passes all E1–E8 evaluations alongside 29/29 tests, with source-integrity checks also passing.

But the number we're proudest of is zero.

Tacit became substantially more autonomous without producing an incorrect autonomous application across the tested submission scope.

We're also proud that our AO integration produced a real engineering result rather than just another tool in the stack.

AO coordinated two dedicated red-team workers and independent reviewers, found a genuine provenance weakness, helped us validate the narrow fix, and the complete safety and regression suite continued to pass afterward.

What we learned

We started this project thinking self-improvement meant making an agent require less human help over time.

We learned that this isn't enough.

An agent that becomes more autonomous by blindly generalizing previous corrections isn't necessarily getting better.

A useful learning agent needs to distinguish between:

I know this.

This looks similar, but I don't know this.

I used to know this, but reality changed.

That last case became one of the most interesting parts of Tacit.

When previously correct behavior changes, preserving a high-confidence memory can be worse than having no memory at all. That's why Tacit explicitly supports weakening and superseding learned policies rather than treating memory as permanent truth.

It also changed how we thought about measuring improvement.

Human reviews fall from 27 to 8 after Tacit learns recurring judgment, but rise from 8 to 11 when we deliberately introduce policy drift during the third close.

At first glance, that looks like regression.

It isn't.

Tacit recognized that reality had changed and correctly gave authority back to the human instead of optimizing for autonomy at any cost.

That led to perhaps our biggest lesson from the project:

Sometimes a self-improving agent gets better by becoming less autonomous.

What's next for Tacit

Finance is our proving ground, not the boundary of the idea.

Cash application gave us a useful environment because exceptions repeat, experienced operators carry significant unwritten knowledge, and incorrect automation has real consequences.

But the same problem exists anywhere experienced teams repeatedly make contextual judgments that never quite make it into documentation.

Operations.

Support.

Compliance.

Procurement.

Internal workflows.

The next step for Tacit is to move beyond a synthetic finance environment and test the learning system against longer-running, real operational workflows.

We also want to expand the evidence and tool sources Tacit can learn from while preserving the principle at the center of the system: models can propose judgment, but autonomy has to be earned through evidence.

Ultimately, we want Tacit to become a learning layer for agents that doesn't just remember what humans did before, but understands when that experience deserves to influence what happens next.

Your team solves the exception once. Tacit remembers how only while the evidence still holds.

Built With

• ao
• css
• html
• javascript
• node.js
• tensormux
• vercel