Aurora-Automation-Support

Monday morning. Same customer. Same $200 order. Fourth time asking for help. And the agent still has no idea what happened before.

Comment

Inspiration

It's Monday morning. A customer emails — their $200 order never arrived, they've contacted support three times already, and each time a different agent gave a different answer. Not because the agents were careless. Because the system has no memory.

Every enterprise support platform treats LLMs as stateless oracles: paste the transcript, get a reply, throw the state away. The agent never knows the customer. It cannot see that this is their fourth contact about the same order, that they're a high-value customer at churn risk, or that the last agent promised a refund that was never actioned.

We built Aurora Support to fix that — not with a better prompt, but with a proper memory architecture.

What it does

Aurora is a fully autonomous e-commerce customer support agent built around a single principle:

CockroachDB is the agent's persistent, queryable, verifiable brain.

Every support request runs through a typed, 6-node LangGraph pipeline:

planner → retrieval → case_state → resolution → guardrail → {escalate | end}

Before the LLM writes a single word, the agent has already:

  • Retrieved the customer's full transaction and conversation history via per-customer vector index (C-SPANN partition in CockroachDB)
  • Pulled the 3 most relevant KB policy passages via <-> distance search
  • Found the most similar previously-resolved case across all customers
  • Computed an autonomy ceiling — act, propose_only, or escalate — from deterministic rules with zero LLM calls
  • Assembled bounded working memory (1800-char budget) so the model only sees what it needs

After the LLM responds, a two-tier guardrail fires:

  • Tier 1 (SQL, no model call): checks groundedness against KB, DB fact consistency (dollar amounts, order status), language match, and autonomy compliance
  • Tier 2 (LLM adversarial critique): fires only on tier-1 failures — with a SHA-256 verdict cache so we never pay twice for the same scenario

Every turn is hash-chained into a tamper-evident audit log. The system scores itself on 5 dimensions per turn with an LLM judge. When scores fall below threshold, it clusters its own escalations, drafts new KB articles, and queues them for human review — a continuous improvement loop that runs automatically.

Key design numbers:
CockroachDB vector indexes on the critical path 3
Guardrail tier-1 LLM tokens cost 0
Autonomy ceiling LLM tokens cost 0
Evaluation dimensions per turn 5
Simulation flywheel deploy gate 85% pass rate
KB policy documents 35+

How we built it

Orchestration: LangGraph StateGraph. Every node is a typed Python function; graph state is checkpointed turn-by-turn into CockroachDB. The standard PostgresSaver uses JSONB SRFs that CockroachDB doesn't support, so we wrote a custom CRDBCheckpointSaver from scratch.

Memory: CockroachDB VECTOR(384) columns co-located with transactional data. Two index shapes:

  • kb_chunks: VECTOR INDEX (embedding) — global, no prefix
  • conversations: VECTOR INDEX (customer_id, summary_embedding) — the prefix column gives C-SPANN a separate k-means tree per customer, so per-customer history search stays O(log n) regardless of total corpus size

LLM: Amazon Bedrock (Meta Llama 3.3 70B Instruct) for all reasoning. Google Gemini as a warm fallback. Circuit-breaker + retry on the Bedrock client; degrades to a clearly-labelled deterministic offline path if Bedrock is unreachable.

Embeddings: sentence-transformers/all-MiniLM-L6-v2 (384-dim) running locally — no external embedding API on the hot path.

Guardrails: Tier-1 is pure SQL + regex (groundedness via <-> distance, DB fact checks, regex for language and autonomy claims). Tier-2 calls Bedrock only on failures, with a guardrail_verdict_cache table keyed by SHA-256 of the decision context.

Simulation flywheel: A SimulatorAgent (Bedrock-powered) plays the customer, generates adversarial tone-driven push-back, and runs the full orchestrator graph with is_simulated=True. The evaluation agent judges each scenario. Deploy is blocked if pass rate is below 85%.

KB gap pipeline: KMeans on escalation embeddings → clusters far from existing KB (distance > 0.4) → Bedrock drafts a new policy article → written to kb_draft_articles as pending_review — never auto-published. A human approve_draft() call embeds it and promotes it to kb_chunks.

CockroachDB — What We Actually Had to Build

Four things nobody warned us about, and one design decision we're genuinely proud of:

  • Custom LangGraph checkpointer from scratch. LangGraph's built-in PostgresSaver calls jsonb_each_text and similar set-returning functions in a FROM clause — a pattern CockroachDB's SQL dialect rejects with no data source matches prefix: jsonb_each_text. We reverse-engineered the BaseCheckpointSaver interface and wrote crdb_checkpointer.py: plain INSERT / SELECT only, checkpoint state serialized as BYTEA via LangGraph's JsonPlusSerializer (not json.dumps — graph state contains UUIDs and LangChain message objects), backed by a ConnectionPool so concurrent API requests don't serialize on a single shared connection.

  • Per-customer C-SPANN partitioning. A flat VECTOR INDEX (summary_embedding) over all conversations would scan every customer's history on every query. The fix: VECTOR INDEX (customer_id, summary_embedding) — the prefix column gives CockroachDB's C-SPANN a separate k-means tree per customer, so search stays O(log n) regardless of total corpus size. This was non-obvious; the index shape came from reading CockroachDB's C-SPANN internals.

  • Embedding format: psycopg vs. VECTOR type. psycopg's default Python list adapter sends arrays as {0.1, 0.2, ...} (Postgres array syntax). CockroachDB's VECTOR type expects [0.1,0.2,...]. Every embedding write needs a to_vector_literal() wrapper. We also hit a double-wrapping bug: one embed_fn was already returning a vector literal string, and a downstream caller wrapped it again — to_vector_literal("[0.1,...]") iterates over string characters and calls float('['), crashing immediately. Fixed by standardizing: embed_fn always returns a plain list; callers own the to_vector_literal call.

  • Guardrail verdict cache as a cost gate. Running a full LLM adversarial critique on every turn is expensive. We store verdicts in a guardrail_verdict_cache table keyed by SHA-256(autonomy_ceiling + failure_category + cited_chunk_ids). Same scenario → same key → tier-2 skipped. This is only possible because CockroachDB stores transactional data, vector indexes, and the verdict cache in the same cluster — a single JOIN is cheaper than a round-trip to a separate cache store.

  • Metadata filter without JSONB SRFs. LangGraph's list() API lets callers filter checkpoints by metadata fields. Standard Postgres would push this into SQL with jsonb_each_text. Since we can't, checkpoint metadata is stored as opaque BYTEA; list() over-fetches on indexed columns and filters decoded metadata in Python. Acceptable at support-chat scale; we documented the tradeoff explicitly.

Challenges we ran into

CockroachDB checkpointing. LangGraph's built-in PostgresSaver uses jsonb_each and similar SRFs that CockroachDB's SQL dialect does not support. We reverse-engineered the checkpointer interface and wrote crdb_checkpointer.py from scratch — serializing the graph state as plain JSONB without any SRF-based extraction.

Per-customer vector isolation. A naive flat index over all conversation summaries means a customer-A query scans customer-B's history. The VECTOR INDEX (customer_id, summary_embedding) prefix design — a separate C-SPANN tree per customer — was the key architectural insight, and it required understanding CockroachDB's C-SPANN partitioning model.

Guardrail cost vs. coverage. Running a full LLM critique on every turn is expensive. The two-tier design — cheap SQL gate first, expensive LLM only on actual failures, with a verdict cache — brings tier-2 cost close to zero in steady state while preserving coverage for edge cases.

Circular trust. Trusting an LLM to decide whether an LLM should take a financial action is circular. The deterministic autonomy_ceiling() function — pure Python rules, no model call — enforces the trust boundary reliably and is auditable by any engineer without reading prompt logs.

Accomplishments we're proud of

  • A memory architecture where every customer interaction builds on every prior one — across sessions, agents, and channels — without a separate vector database
  • A guardrail that catches factual errors and policy violations on the happy path without spending a single LLM token
  • A self-improving loop: the system uses its own escalation failures to automatically draft better knowledge base content, gated by human review before it goes live
  • A simulation flywheel that gates deployment on adversarial scenario pass rates — not just static unit tests

What we learned

  • Persistent memory in customer support is not about storing more — it's about retrieving the right thing under latency constraints, for the right customer, from the right layer
  • Deterministic rules and LLM reasoning are complementary, not competing. Trust decisions (autonomy ceiling) belong in code; language tasks (response generation, article drafting) belong in the model
  • CockroachDB's distributed vector indexing makes it possible to co-locate financial transaction data with the embeddings that describe it — removing an entire architectural tier (the standalone vector store) and the consistency problems that come with it

What's next

  • Real-time streaming responses via WebSocket
  • Multi-channel memory unification (email, chat, voice into one persistent profile)
  • Operator dashboard for live autonomy-ceiling tuning without code changes
  • Production CockroachDB deployment with ccloud provisioning automation
  • Bloomberg / CRM webhook integrations to pre-populate the customer signal profile

Built With

Share this project:

Updates