LabRecall AI

An agent that remembers which research-pipeline repairs actually worked—and learns only from human-confirmed outcomes.

Comment

Inspiration

Computational researchers lose hours solving the same pipeline failure twice. Ordinary troubleshooting chat forgets the environment, cannot distinguish advice from a repair that actually worked, and may confidently repeat a failed action. LabRecall asks a stricter question: what if the memory—not the chat interface—were the product?

What it does

LabRecall turns each failed run into four linked forms of durable memory:

  1. Episodic memory preserves the incident, environment, constraints, and attempted actions.
  2. Semantic memory retrieves analogous confirmed repairs through CockroachDB vector search.
  3. Outcome memory records what a researcher actually tried and whether it worked.
  4. Governance memory records retrievals, recommendations, confirmations, promotions, and evidence IDs.

The agent proposes bounded diagnostics and cites every remembered repair that influenced it. A repair cannot become reusable memory until a person records a confirmed successful outcome. Repeated successes consolidate confidence; confirmed failures reduce it. LabRecall never executes proposed shell or infrastructure changes.

How we built it

CockroachDB is both the transactional system of record and vector store. Incidents, outcomes, repair memories, and audit events share one consistency boundary, preventing operational state from drifting away from embeddings. The live Basic cluster uses namespace-scoped rows, TLS, least-privilege SQL, and two vector indexes.

The public FastAPI app runs on Amazon Lightsail behind rate-limited Nginx and an automatically renewed Let's Encrypt IP certificate. Its fixed IPv4 is the only cloud address in CockroachDB's SQL allowlist. Browser sessions map to hashed namespaces so public tests do not contaminate one another.

The live app currently uses an explicit deterministic embedding provider because AWS is reviewing an account-wide Bedrock Runtime restriction. Provider names are part of CockroachDB namespaces, so fallback vectors cannot mix with future Titan vectors. The public health endpoint exposes the active provider; we never pretend a blocked model call succeeded.

CockroachDB tools used

Distributed Vector Indexing

Normalized embeddings are stored beside transactional memory and queried with namespace-scoped cosine retrieval. This is on the runtime path: remove vector recall and the agent loses its ability to reuse proven repairs.

CockroachDB Agent Skills

The official Agent Skills repository is pinned as a Git submodule. Its transaction, SQL, and cloud-security skills were used as reproducible engineering review gates. The findings and fixes are preserved in docs/SKILL_EVIDENCE.md.

AWS service used

Amazon Lightsail hosts the functional public agent environment with fixed egress, HTTPS, health/readiness probes, and instance monitoring. Bedrock Titan and Nova adapters are also implemented, while successful live inference is deliberately not claimed until AWS removes the account restriction.

Challenges

CockroachDB Cloud rejected a managed-cluster setting unavailable to application users, so the migration now relies on the platform's current vector support. We also treated retries as a correctness problem: serialization failures replay a short transaction with jitter, while ambiguous commits are surfaced for operation-ID inspection instead of being blindly replayed.

AWS Bedrock failed identically through Boto3, direct bearer HTTPS, two regions, and the account-owner Playground. Instead of hiding the failure or blocking the demo, we isolated a deterministic provider, documented the evidence, and opened an AWS Support case.

Accomplishments

  • Real write → retrieve → confirmed outcome → promotion → learned recall on CockroachDB Cloud.
  • Public HTTPS proof recalled the confirmed repair at cosine similarity 0.823063.
  • Four memory types share one transactional, vector-capable database.
  • One-click proof shows cold abstention, human promotion, paraphrased recall, provenance, confidence, and an ordered audit trail.
  • Public service includes bounded inputs, request IDs, safe errors, security headers, throttling, concurrency limits, deterministic degradation, and readiness checks.
  • 21 automated tests and Ruff pass from a clean public repository.

What we learned

Production memory is not a transcript. It is evidence with provenance, outcomes, confidence, isolation, and a policy for when experience may affect the next decision. CockroachDB matters because the operational event and semantic memory do not cross an eventual-consistency boundary.

Try it

Open the public demo, select Run the 20-second proof, and inspect the evidence card and governance timeline. The demo uses synthetic data only; do not enter personal, confidential, or proprietary information.

What's next

Organization-scoped authentication, richer outcome calibration, multi-researcher evaluation, reviewer queues, automated retention controls, and switching the isolated provider namespace to Titan after AWS Support enables Runtime inference.

Built With

Share this project:

Updates