Our main Dashboard

What's next for Pin-On-Expand

Read the receipt

Inspiration

Paritok's roadmap has an open item: adaptive compression — per-segment auto-selection of aggressiveness based on age, kind, and downstream intent.

I went looking for a signal that could drive it, and found one hiding in plain sight. When a compressed agent context isn't enough and the model calls expand_context, it is telling you that compressing that content was a mistake. Not guessing. Stating it, in a tool call, with the exact reference id.

That is free supervision, generated by the system itself, at no cost, on every failure. And nothing consumes it.

I set out to build a policy that listened to it. To prove the policy helped, I first had to measure what stock Paritok cost. That measurement is where the project actually went.

What I found instead

Building the harness meant counting every HTTP POST the proxy made upstream. The moment I did, the numbers stopped agreeing with each other.

Paritok's /stats said I had saved 64% of my input tokens.

The provider had been billed 69% more than if I had sent the file with no compression at all.

Same session. Same twenty thousand tokens. Both numbers correct.

The mechanism

When the model calls expand_context, the proxy answers it itself: it appends the full original file to a proxy-local thread and POSTs that thread upstream a second time.

stats is computed once, in process_request — before that loop runs.

post 0:   6,919 tokens   compressed request          ← counted by /stats
post 1:  26,924 tokens   carries the full original   ← never counted
                         -------------------------
billed:  33,843

There is a second effect, and it is the one that makes this compound. _conceal_virtual_calls strips the exchange from the reply, so the agent never sees it. Next turn the agent re-sends the original, Paritok re-compresses it to the same reference, and the model expands it again. Every turn. Forever.

To be fair to Paritok: their README says /stats is "scoped to what Paritok actually intervenes in." This is not a hidden number. But the excluded traffic is generated by the gateway itself — which makes it the one category a user would most expect to see counted.

The arithmetic

This is where it stopped being a bug report and became something more interesting.

Let:

T = tokens in the original file
c = compression ratio = compressed size / original size (0 < c < 1)

Measured on the hosted 4B model:

c ≈ 0.36

A turn where the model does not expand

C_quiet = cT

You pay only for the compressed context.

A turn where the model does expand

C_expand
    = cT + (cT + T)
    = T(1 + 2c)

You pay for:

one compressed request
another compressed copy
one full original file

Now compare that with simply sending the original:

Cost without compression = T

The ratio becomes

C_expand / T = 1 + 2c

Since

c > 0

then

1 + 2c > 1

Therefore, on any turn where the model expands, compression cannot win — no matter how good the compressor is.

That result does not depend on Paritok. It is not a bug in their model. A perfect compressor, one that squeezed context to a single token, would still lose on an expanding turn, because the expansion re-sends the original anyway and the compressed copy rides along as pure overhead. The loss is structural, and it lives in the gateway's control flow rather than in its weights.

The two numbers users see also come from the same c.

Reported saving:

1 − c

Actual overspend:

2c

For the measured value

c = 0.36

this predicts

reported saving = 64%
actual overspend = 72%

Measured:

reported saving: 64.0%
actual overspend: 69.2%

The remaining difference comes from tag and message-structure overhead that the simple model ignores.

Two true statements about one session, pointing in opposite directions.

The fix

If an expand_context call is the model stating that compression was wrong there, then honour it: pin that content and pass it through verbatim from then on.

The reference never reappears, so there is nothing left to expand, and the turn costs one POST instead of two.

Pins are keyed by content hash and source path, so re-reading the same file at a different offset still hits. Content nobody expands is compressed exactly as before. It narrows compression only where the model has already demonstrated that compression failed.

Over n turns where the model needs exact source every turn:

Stock Paritok:

C_stock(n) = nT(1 + 2c)

Pin-on-expand:

C_pin(n)
    = T(1 + 2c) + (n − 1)T
    = T(n + 2c)

Stock pays the doubled cost every turn.

Pinning pays it once, then settles at simply sending the original.

The saving from pinning is

Saving
    = 1 − C_pin / C_stock
    = 1 − (n + 2c) / (n(1 + 2c))

As the number of turns grows,

Saving → 2c / (1 + 2c)

For

c = 0.36

the theoretical limit is

41.9%

n = 3

the model predicts

27.9%

Measured on two real proxy processes against the hosted GPU:

29.1%

3 turns, hosted GPU	POSTs	Billed	vs no proxy
stock Paritok	6	104,229	+73.7%
pin-on-expand	4	73,901	+23.1%

The control case—where the model never expands—is a dead heat at 20,787 billed tokens each.

Pinning costs nothing when it never fires.

The theory predicts the measurement to within about one percentage point, which is the part I am most pleased with. The arithmetic and the proxies were built independently and they agree.

What this does not fix

Pinning does not make that session profitable, and I am not going to pretend it does.

It moves a 73.7% overspend to 23.1%.

The remaining overhead is exactly what the model predicts.

Average cost per turn:

C_pin / (nT) − 1
    = 2c / n

n → ∞

this approaches

The first expansion always has to happen before the pin exists, so the first turn is always paid in full.

Pinning converges to break-even from above.

It never beats sending the original file directly in a session where the model genuinely needs the original.

That is the honest ceiling, and the honest goal is to stop losing badly rather than to claim a win that isn't there.

What I got wrong

Almost everything, first time. The finding survived only because the harness kept catching me.

My first fixture had 26 near-identical functions, which Paritok's deduplicate_definitions collapsed to nothing—a glorious, fake 95% compression rate.

My SEG-parsing regex matched the literal [SEG] in the prompt's instruction text rather than the actual segment, so everything compressed to ....

I used Paritok's own wrapper.py as a fixture without noticing it contains the strings expand_context and [REF:, which fired my detection logic on the fixture's own text.

And the proxy's in-memory cache outlived a fix, making a working correction look broken.

Three of those four would have shipped a confidently wrong headline.

Worse: my first reproduction was right about the wrong thing.

I confirmed an expand-then-re-collapse loop, then realised _conceal_virtual_calls means that only applies to SDK mode.

In proxy mode the failure is different—an accounting gap plus per-turn re-expansion.

I threw the model out and re-derived it from server.py.

Every serious bug I hit was in my own measurement, not in Paritok.

That is the lesson I am taking:

When you are making a claim about someone else's numbers, the infrastructure that proves you are not lying is the deliverable.

The fix is about 120 lines.

The harness that earns the right to publish it is most of the repository.

Try it yourself

https://pin-on-expand.onrender.com

Paste any source file.

It cold-starts two real paritok proxy processes—one stock and one patched—drives a genuine multi-turn session through each, and shows you the reconciliation:

every upstream POST
which ones /stats counted
what the provider actually billed

Compression runs on Paritok's hosted 4B model.

Nothing on that page is hard-coded.

Every run cold-starts its proxies because the compression cache, shadow store and pin set all live for the lifetime of the process. A warm proxy would silently hand back a previous run's answer.

What's next

The level parameter in Paritok—L0 through L3, mapping to real target compression ratios from ≤0.50 down to ≤0.20—is fully implemented and wired to the model.

Nothing ever selects between values.

Tool results always take the default L1.

History is hardcoded to L3.

Pin-on-expand is only the binary case of the policy I originally wanted:

not compress or don't,

but

how aggressively should this segment be compressed, given its age, type, and relevance to what the user is actually asking?

And bge-small is already a dependency for tool discovery, so intent-similarity scoring needs no additional packages.

Paritok already built the dial.

It just doesn't have a controller yet.

That is still the thing I originally set out to build.