Agent-Guard

(1) Landing Page
(2) Sign Up Page
(3) Dashboard - Incident Console
(4) Incident Console - Error Detected
(5) Incident Console - Diagnosing
(6) Incident Console - Diagnosed
(7) Incident Console - Remediating
(8) Incident Console - Resolved
(9) Incident Console - Resolved logs
(10) Incident Console - Resolved logs
(11) Lambda Function
(12) Cloud Watch Logs - Before Remediation
(13) Cloud Watch Logs - After Remediation

Inspiration

Kubernetes environments are increasingly watched by AI agents instead of humans staring at dashboards. The obvious way to scale that is to spin up more agents — but naive fleets of agents don't share memory, so they duplicate work, contradict each other, or race to fix the same thing twice. A cost-approval agent can approve a rollback five seconds after another agent already spent the same budget on a scale-up. None of these agents are individually wrong — they're wrong together, because nothing arbitrates shared state between them. We wanted to build the system where that failure mode is structurally impossible, not just less likely.

What it does

AgentGuard is an incident-response system where multiple AI agents investigate and fix pod/service failures concurrently, all sharing one transactional memory backed by CockroachDB.

Concurrent diagnosis: 8 LangGraph agents investigate the same incident at once, each pulling similar past incidents from a CockroachDB vector index. A serializable transaction reconciles their conclusions — exactly one root cause is ever CONFIRMED, the rest are SUPERSEDED, never left as silently contradicting "truths."
Concurrent remediation: once a root cause is confirmed, N remediation agents race to claim and execute the fix. An atomic conditional UPDATE ensures only one claims it; a write-skew budget transaction on a shared namespace budget causes and correctly retries real SQLSTATE 40001 conflicts under concurrent draws.
Real AWS failures, not scripted ones: we deployed agentguard-pod-worker, a real AWS Lambda function standing in for a pod's container process. Clicking "Invoke → Crash / OOM Kill / Timeout" causes a genuine AWS failure (Runtime.OutOfMemory, Sandbox.Timedout, an unhandled exception) — the agents diagnose the actual AWS error text, not a canned string.
Real remediation, not a status flip: the winning remediation agent calls AWS's UpdateFunctionConfiguration for real — raising Lambda memory for an OOM diagnosis, raising the timeout for a Sandbox.Timedout diagnosis, or deploying a hotfix environment variable for a CrashLoop diagnosis — then re-invokes the same function to verify the fix actually holds before marking the incident resolved.
An independent Consistency Checker audits agent_actions and final table state after every run: no duplicate commits, no negative budget, no incident left inconsistent.

How we built it

Sequential relay across three build phases, each handing a working package to the next:

Shared memory foundation — CockroachDB Cloud cluster, @agentguard/db (typed client, connection pooling, SQLSTATE 40001 retry wrapper), the incident_memory vector table, and a pod-event simulator.
Concurrent diagnosis — a LangGraph subgraph fanning out N diagnosis agents via Send, a reconciliation transaction that checks for an existing CONFIRMED diagnosis inside the same transaction as the write, and a benchmark proving the siloed-vs-reconciled contrast.
Concurrent remediation, AWS integration, and delivery — the atomic claim transaction, the budget write-skew transaction, the ccloud CLI–backed Ops/Capacity agent, a full-fleet benchmark (10/50/100 concurrent agents), the Consistency Checker, the real Lambda chaos-and-fix loop, and the live console tying diagnosis + remediation + consistency together in one dashboard.

Stack: React/Vite frontend, Express + LangGraph backend, Gemini for embeddings (3072-dim) and agent reasoning, CockroachDB Cloud for shared transactional + vector memory, AWS Lambda for the real monitored/remediated workload, Supabase for auth and session state.

Challenges we ran into

Proving the concurrency claim instead of asserting it. It's easy to build an agent with a vector-search memory and call it "agentic memory." The bar we set for ourselves was: removing CockroachDB's serializable guarantees should visibly break the app. That meant deliberately engineering real races — concurrent diagnosis proposals on the same incident, concurrent budget draws on the same namespace — and catching genuine 40001 retries under load, not just writing code that looks correct.
Making a "pod crash" real instead of scripted. Our original design (like most hackathon submissions under time pressure) would have been a random string generator writing fake failure types into a table. We instead deployed a real, deliberately fragile Lambda function and tuned its memory/timeout so oom and timeout modes are genuine AWS-enforced kills, not simulated ones — and had to redesign the chaos payload from an unbounded allocation loop (which crashed in a way that didn't cleanly reverse after a memory increase) to a bounded, parameterized allocation so the same invocation could be shown failing before a fix and succeeding after it.
Making the fix real too. The natural failure mode was letting remediation only flip a database status. We pushed further: the winning agent makes a real UpdateFunctionConfiguration call against AWS and then re-invokes the function to verify the fix actually holds — so "RESOLVED" means a real AWS state change happened, confirmed by a second real AWS call, not just a database write.
Observability gaps that looked like bugs but weren't. UpdateFunctionConfiguration never appears in CloudWatch Logs — only in CloudTrail — so it initially looked like remediation "wasn't doing anything." We fixed this by tagging the verification re-invoke with a verify: true flag the handler logs explicitly, so the fix's effect is visible where you'd naturally look for it.
TypeScript/zod/LangChain version friction — DynamicStructuredTool's generic inference from a ZodObject blew up TypeScript with TS2589 (excessively deep instantiation) on this project's dependency combination. Pinning zod lower "fixed" the type error but broke at runtime, since @langchain/google-genai requires a subpath export zod only added later. The real fix was an explicit param type plus a narrow as any cast on the schema only — no runtime change.

What we learned

That "agentic memory" only means something when removing it breaks the system. Building two independent, real concurrency mechanics — diagnosis reconciliation and a remediation race — forced us to actually use CockroachDB's serializability rather than just storing data in it. We also learned that credibility compounds: real Lambda crashes make the diagnosis agents' work meaningful, and a real AWS fix (verified by a real re-invocation) makes the remediation agents' work meaningful in a way a database status field never could.