PipelineGuard — AI Data Pipeline Triage Agent

The Problem That Inspired Me

As a Solutions Architect working with data teams every day, I see the same painful pattern: when a cloud data pipeline fails, engineers lose hours digging through logs, hunting for runbooks, and writing tickets. This creates delays, outages, and burnout. I wanted to build something that turns this messy internal work into seconds of reliable automation.

What I Built

PipelineGuard is a fully tool-driven agent inside Elasticsearch Agent Builder that:

1. Uses a custom ES|QL tool to instantly find HIGH-severity errors in time-series pipeline logs
2. Automatically calls a vector search tool (semantic_text) to pull the exact fix from internal engineering runbooks
3. Outputs a clean, professional Markdown triage report with root cause, step-by-step fix, estimated time saved, and execution order

How I Built It (in one afternoon)

• Created two indices in Kibana Dev Tools (data-pipeline-logs + engineering-runbooks)
• Built Tool 1: check_high_severity_logs (ES|QL)
• Built Tool 2: search_runbooks (vector search)
• Created the PipelineGuard agent with precise instructions so it always calls tools in the right order
• Tested live in the Agent Builder chat

Challenges & What I Learned

The biggest challenge was writing instructions so the agent reliably calls both tools in sequence without hallucinating. I solved it by being extremely explicit in the system prompt.

I learned how incredibly fast and powerful Agent Builder is — you can go from idea to production-grade agent with zero external code or frameworks. The native ES|QL + semantic search integration feels like magic.

Why This Matters

PipelineGuard directly addresses the hackathon themes: automating messy internal work, tool-driven agents, measurable impact (45 min – 2 hrs saved per incident), and time-series aware reasoning. It proves Elastic is making practical AI agents accessible to every engineering team today.

Watch the 3-minute demo to see it in action!

Built With

• agent-builder
• dev
• elasticsearch-serverless-(observability-complete)
• es|ql
• kibana
• semantic-text-vector-search