CASSANDRA

Inspiration

AI agents can confidently give wrong answers. We wanted to build a system that automatically watches AI agents, finds mistakes, and helps fix them without relying on humans to check logs.

What it does

Cassandra monitors AI agent conversations, detects problems like hallucinations and tool failures, finds their root causes, creates tests, improves the agent's prompt, and checks whether the fix works.

How we built it

We built Cassandra using Gemini, Google ADK, Vertex AI, Arize Phoenix, MCP, Python, React, Cloud Run, and Firestore. Phoenix provides the agent traces, while Cassandra analyzes and improves them.

Challenges we ran into

We faced issues with experiment support, hidden errors, self-monitoring loops, and making sure prompt fixes actually improved the agent instead of solving only one problem.

Accomplishments that we're proud of

Cassandra can detect a failure, diagnose it, suggest a fix, replay the original question, and run tests against the updated agent in under a minute. It can also monitor its own performance.

What we learned

We learned that AI agents need continuous monitoring and testing. A single prompt fix is not enough. Good observability and repeated evaluation are essential for reliable AI.

What's next for CASSANDRA

We plan to support more types of AI failures, automatically deploy safe fixes, monitor multiple agents, add long-term analytics, and make Cassandra capable of supervising entire AI agent fleets.