• 

  INSTAR

INSTAR

Agent skills that learn from their own failures — in a database where two agents cannot teach them opposite things. An instar is the stage of a cockroach's life between two moults.

Inspiration

Agent skills are markdown files that never change. Every code review finds the same mistake again; every failure is thrown away. The obvious fix — write what you learned back into the skill — breaks the moment two agents do it at once, because one agent's "always use X" is another's "never use X" and nothing in a database schema notices. That failure mode, not the writing, is the interesting problem.

What it does

INSTAR stores lessons attached to slots inside a skill (tenant, skill, slot, polarity ±1, status). Retrieval is a prefix-filtered ANN scan. Status transitions today are candidate → contested, driven by the conflict path; promotion to canonical is designed and not implemented. The invariant: for any (tenant, slot), the active lessons must not contain both a +1 and a −1 without an open conflict record. That is a predicate over a set — no UNIQUE index, CHECK, or exclusion constraint can express it, because none can range over other rows. Only transaction isolation protects it.

Corpus: 2,353 rows, all real, zero synthetic — 50 skill versions (15 BenchFlow, skill-creator, 34 from cockroachlabs/cockroachdb-skills), 2,027 documentation chunks, and 276 lessons mined from 42 real review documents (13 BLOCK / 129 FIX / 134 NIT across 274 slots). Embedding the lot cost $0.0039 (1,569 Bedrock calls, 193,630 tokens).

The falsification experiment

npx tsx src/demo/storm.ts --agents N — N agents fighting over 6 contested slots, same cluster, same code, same workload. The only difference between the two arms is one SET TRANSACTION ISOLATION LEVEL.

Run at two scales, because a single run of a concurrency experiment proves very little. The structural result is identical at both; only the magnitudes move.

	RC · 8 agents	SER · 8 agents	RC · 16 agents	SER · 16 agents
undetected contradictions	5 of 6 slots	0	5 of 6 slots	0
ledger drift	41	0	92	0
lessons stored	43	12	94	12
open conflicts filed	2	6	1	6
retries absorbed	6	124	1	405
failed writes	0	0	0	0
elapsed	2.4s	9.7s	2.0s	19.3s
integrity	CORRUPTED	CLEAN	CORRUPTED	CLEAN

Read the columns in pairs. Under READ COMMITTED the library ends up believing both "do X" and "never X" on 5 of 6 topics with no record that it disagrees with itself, at either scale. Under SERIALIZABLE it is 0 and 0 at either scale, and it converges on the same 12 lessons and 6 filed conflicts regardless of how many agents were shouting — which is the actual point: the outcome stops depending on the concurrency.

Which numbers you can check where. The 8-agent run is the one in the demo video, and its state is what the live console audits — open the demo, scroll to "2 · Why this needs CockroachDB", and those four numbers per arm are queried from the cluster on page load, not read from a log. The 16-agent run is committed verbatim at docs/storm-run-2026-08-12.txt. The video uses 8 because the 16-agent SERIALIZABLE arm takes 19.3s of real cluster time, which is too long a silence for a 3-minute video — not because it behaves differently.

Counts drift between runs; that is what a live experiment does. "5 of 6 versus 0" has not moved.

The argument is not "CockroachDB has SERIALIZABLE" — Postgres has it too. It is that correctness here depends entirely on isolation, CockroachDB makes the safe choice the default, hundreds of aborts were absorbed with zero failed writes, and no constraint will ever warn you that you needed it.

CockroachDB tools — and what the agent actually did with them

• Distributed Vector Indexing. Three vector indexes, VECTOR(256), vector_cosine_ops, prefixed on (tenant_id, status). The plan shows • vector search / table: lesson@lesson_vec / prefix spans: [/'ed01fe73…'/'candidate' - …] — both predicates evaluated inside the ANN scan. pgvector's HNSW has no equivalent: you post-filter (ask for LIMIT 8, get fewer usable) or maintain one partial index per value. Live recall 136ms through the API, ~45ms direct.
• ccloud CLI. Cluster provisioning, SQL user creation, connection string, and scripts/ru_meter.sh, which reads the RU meter from the billing API — because crdb_internal.tenant_usage_details is empty on Basic and needs SET allow_unsafe_internals = true just to read.
• Cloud Managed MCP Server. Registered, OAuth'd, read-only, pinned to one cluster ID. Deliberately the human audit surface, not the app's data plane: it caps responses at 10 KiB and has no update_rows/delete_rows. Judges can inspect the ledger without being able to change it.
• Agent Skills Repo. All 34 sponsor skills ingested as live corpus.
• AWS Bedrock — amazon.titan-embed-text-v2:0, {"dimensions":256,"normalize":true}, measured returned L2 norm 1.000000005. IAM user scoped to Bedrock only, $0.01 zero-spend alarm. Lambda/S3/Secrets Manager were designed but the shipped demo uses Bedrock only; we do not claim services we did not use.

Challenges we ran into

A covering index cannibalised the vector index. CockroachDB's own optimizer recommends CREATE INDEX ON lesson (tenant_id, status) STORING (body, embedding). Create it and the plan collapses to a plain scan — the ANN path never runs. Results stay correct. Nothing warns you.

A parser that found 92 of 276 findings while looking perfectly healthy. The 42 review documents use five distinct finding syntaxes. Handling only the first dialect yielded plausible output. It was caught only because the reviews declare their own counts — 11 files declared 6 findings and parsed 0. The gate now checks declared-vs-parsed: 40 of 42 match, 0 mismatches; 2 declare no tuple and are logged rather than quarantined (strict quarantine would drop 12 real findings).

Vector-insert concurrency: less is more. 16 writers ≈38 rows/min; 4 writers ≈123 decaying to ≈18; index dropped, 12 writers ≈381. Two limits, the first masking the second: C-SPANN partition contention (every row shares one prefix during bulk load), then Bedrock throttling on a new account. We guessed wrong twice — Bedrock, then the connection pool — before SHOW QUERIES showed all 16 workers on one INSERT, the longest running 43 seconds. Fix: the documented pattern — seed, then index.

A hot row we built ourselves. spend_ledger had one row per day, touched twice per item by every worker; throughput fell to ~20 rows/min and the ledger logged 200 Bedrock calls while 98 rows had landed. The fix was never weaker isolation — it was not creating a hot row.

What we learned

Cost: ≤39.4 RU per vector insert (upper bound including deletes and meter reads; insert-only 26.1). The full 2,353-row corpus is 0.19% of the free 50M cap; 50 careless re-ingests would be 9.3%. Also: a vector-indexed column accepts NULL once NOT NULL is dropped, and such rows are simply absent from the index with the plan unchanged.

Honesty rails

We claim no measured performance lift. A prior benchmark corpus exists but its ci95 straddles zero (n=1–3 per task) and the raw rollouts are gone; it is cited as provenance only. Documentation chunks are retrievable context, not learned lessons — slot doc:*, permanently candidate, never promoted, enforced by a post-load assertion. And the free cluster is not reproducible from bootstrap.sh: no-card Basic is Console-UI-only, one per organisation.

Not built, stated plainly

The README carries the same list, and the pitch should not be more flattering than the repo:

• The episode table has no writer — every lesson today comes from a review document, not an observed agent run.
• The holdout arm exists in the schema (injection.arm IN ('treatment','holdout')) and no code writes it. No utility measurement is claimed, because none exists.
• Promotion to canonical, confidence and decay are columns with defaults and no scheduler.
• AWS Lambda, S3 and Secrets Manager were designed into the architecture and are not used. Bedrock is the only AWS service INSTAR calls.
• The free CockroachDB cluster is not reproducible from a script — the no-card Basic cluster is Console-only, one per organisation. The migrations are the reproducible part.

Upstream contribution

Not required by the rules, and done anyway: a skill was contributed back to the sponsor's own repository — cockroachlabs/cockroachdb-skills PR #21, Add tuning-vector-index-workloads skill (opened 2026-08-12, 186 lines), written out of what this build actually measured about C-SPANN partition contention and the seed-then-index ordering. It is open and unmerged — whether it lands is the maintainers' call.

What's next

Mine the remaining lessons into further upstream skills; a conflict-resolution review queue; multi-region prefix routing.

Built with

cockroachdb · cockroachdb-cloud-basic-v26.2.5 · distributed-vector-index · ccloud-cli · cockroachdb-mcp-server · aws-bedrock · amazon-titan-embed-text-v2 · typescript · node · pg · agent-skills · serializable-isolation

---

Built solo during the submission period with Claude (Anthropic) as an AI coding assistant for code generation, debugging and documentation; all pre-existing code and third-party libraries used are listed in the repository README.

Built With

• agent-skills
• amazon-titan-embed-text-v2
• aws-bedrock
• ccloud-cli
• cockroachdb
• cockroachdb-cloud-basic-v26.2.5
• cockroachdb-mcp-server
• distributed-vector-index
• node.js
• pg
• serializable-isolation
• typescript