Throughline

Inspiration

At 2 a.m., mid-incident, an agent that answers "no prior incidents" might be telling you the archive is empty. It might also be telling you the embedding call timed out. Those are opposite facts, and most memory systems return them as the same string. Three silent failures inspired this build: facts that quietly stop being true, absence of evidence reported as evidence of absence, and eviction policies that eat the newest memory first, because the incident that just happened has no usage history yet. All three are engineering problems, not prompt problems.

What it does

Throughline is an incident-response agent whose memory you can audit, built for on-call engineers who can't afford to trust a black box. Every recall carries a receipt: which retrieval path actually ran, how many candidates were examined and how many came back, what was excluded and under which rule, and a coverage verdict. If the search could not run, the verdict is UNKNOWN, and a boundary guard makes it an error to turn UNKNOWN into "there's nothing".

Memories are typed, and the type is load-bearing. "The primary is db-7" is an entity fact. It's true on Tuesday and dangerous on Friday, so it decays in 14 days. "Restarting the pods did not help" is a rejected hypothesis, and it keeps its value for a year, because knowing what didn't fix an outage is half the point of keeping an archive. Contradicting a fact doesn't overwrite it: the old row gets an end date and a link to what replaced it, so "why did you tell me that in June" is a question this archive can answer. And ranking is deterministic. Similarity, decay, confirmations and contradictions combine into one score computed in code. Claude writes the narrative around the results and never produces the number that ordered them.

How I built it

A TypeScript monorepo with the memory layer as a pure package: no web framework, no cloud SDK, importing anything from the app layer is a build error. Around it, a Hono API on Node 22 runs the agent loop, and an Astro + Preact console renders the archive and the receipts verbatim.

The memory lives in CockroachDB Cloud, recalled through a distributed vector index over a VECTOR(1024) column. Two findings came from measuring instead of reading documentation. Vector indexing works on the free Basic tier, which I couldn't find documented anywhere. And an index on the embedding column alone is useless for this workload: the planner accelerates a filtered nearest-neighbour query only when the filters match the index's prefix columns, so the real index is (workspace_id, is_live, embedding), with is_live as a stored computed column. A capability probe reads the live query plan to prove the planner actually chooses it.

The agent runs on Amazon Bedrock in eu-central-1. Titan (amazon.titan-embed-text-v2) embeds the queries and Claude answers over the Converse API, with both model ids read off the live account rather than documentation. CockroachDB's managed MCP server sits outside the hot path as an independent verification channel that has to agree with the direct one, because an agent shouldn't grade its own homework. Provisioning is scripted through the ccloud CLI, and the deploy target, Lambda behind CloudFront with secrets in Secrets Manager and EventBridge driving scheduled eviction, is described in CDK. Because a public Bedrock demo is a public spend surface, the API carries a per-client rate limiter and a daily call ceiling counted in the database.

Challenges I ran into

The one that could have poisoned the demo silently: two vector spaces in one column. The demo rows were first seeded with the local deterministic embedder while recall went through Titan. Cosine similarity between vectors from two different embedders is a meaningless number, and nothing throws. Everything got re-seeded through Bedrock, and the seeding and verification CLIs now refuse to run with a different embedder than recall uses.

The vector index that did nothing. The first index covered the embedding column alone, and the planner full-scanned every query this system actually runs. Finding that meant reading query plans, not documentation, and the capability probe now does that permanently.

Bedrock model access has a trap: an id can appear in list-foundation-models and still refuse on-demand invocation, demanding an inference profile. Model ids are proven with billed calls before anything depends on them.

Accomplishments that I'm proud of

Recall answers you can audit, and a refusal guard that was caught working on camera: break the embedder on purpose and the agent says UNKNOWN instead of inventing an absence. A vector index the planner provably uses, on the free tier. And a README whose status section stays honest, with dates on every measurement: over a thousand tests guard the memory layer, many written specifically to go red if a protection is removed.

What I learned

Measure, don't assume. The undocumented free-tier capability, the prefix-column planner rule and the inference-profile trap all came from live measurement. The quieter lesson: keeping written claims true while a system changes under them is real engineering work. A test count in the README went stale twice within one afternoon, and the video script claimed an embedding path that a live check disproved. The discipline that survived: measure, date the measurement, and say UNKNOWN when you can't check. Which is what the product does.

What's next

Embedding agent-stored rows at write time, so facts typed mid-incident join recall coverage immediately. Standing up the CDK-described deploy for a public demo URL. And a service account key for the managed MCP server, so the verification channel's lamp turns green next to the rest.

Built With

• amazon-bedrock
• amazon-cloudfront
• amazon-eventbridge
• amazon-titan
• amazon-web-services
• astro
• aws-cdk
• aws-lambda
• aws-secrets-manager
• claude
• cockroachdb
• hono
• model-context-protocol
• node.js
• preact
• typescript