Data Reliability Agent

Inspiration

The inspiration for the Data Reliability Agent came from my experience in computer architecture and senior design courses. While working with assemblers and RISC-V architecture, I struggled with processor pipelining. I spent many hours tracing stages and dependencies, trying to understand what had gone wrong and where I should investigate next.

Although processor pipelines and data pipelines are not the same, that experience stayed with me. In both cases, knowing that a pipeline failed is only the beginning—the difficult part is determining why it failed.

That frustration inspired us to build the Data Reliability Agent: a tool that helps engineers diagnose pipeline failures using verified evidence, knowledge preserved from previous incidents, and AI-generated guidance grounded in that evidence.

What it does

The Data Reliability Agent converts a failed pipeline run into a structured incident. It displays deterministic rule violations and operational evidence so engineers can see exactly what failed.

The application creates a vector representation of the incident and stores it in CockroachDB. When a new incident occurs, CockroachDB Distributed Vector Indexing retrieves semantically similar historical incidents using cosine similarity.

These relevant memories are supplied to Amazon Bedrock as grounded context. Bedrock uses the current evidence and historical incidents to produce:

A clear explanation of the failure
Possible causes
Recommended next steps
A confidence assessment

The verified evidence always remains authoritative. The AI helps interpret the evidence, but it cannot replace or contradict the deterministic findings.

How we built it

We built the project with a serverless architecture on AWS. AWS Lambda runs the backend logic, while Amazon API Gateway exposes the application endpoints. Amazon Bedrock generates the grounded incident analysis.

CockroachDB serves as both the operational database and the agent’s long-term memory layer. Each incident is stored with its structured evidence and a 256-dimensional embedding. CockroachDB Distributed Vector Indexing allows the application to retrieve similar historical incidents efficiently and provide them to Bedrock as context.

Amazon S3 and Amazon CloudFront host and deliver the frontend. We also used Amazon CloudWatch for monitoring, AWS Identity and Access Management for permissions, AWS Systems Manager Parameter Store for configuration, and AWS SAM with AWS CloudFormation to define and deploy the infrastructure.

Challenges we faced

One of our biggest challenges was combining deterministic validation with generative AI safely. We did not want the model to invent a diagnosis or replace facts established by the application. We addressed this by clearly separating verified evidence from AI-generated interpretation and grounding every response in the current incident and retrieved historical memories.

Another challenge was designing useful vector memory. We had to determine which incident details should be embedded, how similar incidents should be ranked, and how retrieved memories should be presented so users could understand why they were relevant.

We also had to coordinate multiple cloud services while keeping the application reliable, secure, and responsive.

What we learned

We learned that AI is most useful for reliability work when it is grounded in structured evidence and previous operational experience. Vector search gives an agent more than storage—it gives it a practical form of memory.

We also learned how CockroachDB can combine transactional incident data with semantic vector retrieval in one system, and how Amazon Bedrock can transform that retrieved context into understandable and actionable guidance.

Most importantly, this project changed how I think about the pipeline problems I encountered in computer architecture. Instead of treating every failure as an isolated debugging exercise, we can preserve what was learned and use that knowledge to make the next failure easier to solve.