Inspiration

Modern businesses face operational issues across incident reports, logs, support tickets, and internal documentation. Resolving these issues often requires manually gathering evidence, identifying the root cause, coordinating actions, and verifying outcomes. We were inspired to build an AI system that goes beyond generating recommendations and helps teams execute operational workflows safely, transparently, and efficiently.

What it does

OpsPilot is an AI-powered operations agent that transforms incident reports into structured, evidence-backed workflows. It retrieves relevant information from approved documents, develops an execution plan, invokes authorized tools, and verifies the results.

Key features include:

Intelligent incident analysis: Understands and classifies incoming operational issues.

Evidence-grounded recommendations: Connects recommendations to relevant documents, logs, and tool observations.

Autonomous workflow execution: Uses tools to perform approved operational tasks.

Human-in-the-loop approvals: Requires human authorization for consequential actions.

Outcome verification: Checks actual execution results instead of assuming success.

Auditable execution: Maintains a record of decisions, actions, approvals, and outcomes.

How we built it

We designed OpsPilot using a modular architecture that combines AI reasoning, document retrieval, controlled tool execution, and persistent workflow management.

Frontend: React and TypeScript for incident submission, workflow visualization, and approval management.

Backend: Python and FastAPI for REST APIs, request validation, and workflow orchestration.

Database: PostgreSQL for incidents, workflow states, execution records, and audit events.

Knowledge retrieval: Document processing and semantic search to retrieve relevant operational procedures and supporting evidence.

AI integration: A tool-calling-capable language model for incident analysis, structured planning, and tool selection.

Safety and verification: Permission checks, approval gates, validated tool inputs, and independent post-action checks.

Testing and deployment: Automated API tests and a reproducible development environment.

The system is designed to separate AI-generated decisions from deterministic safety controls, ensuring that the model cannot independently bypass authorization requirements.

Challenges we ran into

One of our primary challenges was designing an agent that could execute multi-step workflows without becoming unpredictable or unsafe. Language models can generate incorrect assumptions, select inappropriate tools, or report success without sufficient evidence.

We addressed these design challenges by introducing structured workflow states, explicit tool contracts, evidence references, permission checks, and independent outcome verification.

Other challenges included:

Integrating the frontend, backend, database, and AI components.

Retrieving relevant evidence from operational documents.

Managing tool failures, retries, and interrupted workflows.

Balancing autonomy with human oversight.

Keeping the project achievable within the hackathon timeline.

Accomplishments that we're proud of

We are proud of designing OpsPilot around a principle that is often overlooked in AI applications: an AI-generated answer is not the same as a verified outcome.

Our key accomplishments include:

Designing an end-to-end architecture for evidence-driven operational automation.

Combining AI planning, knowledge retrieval, tool execution, and verification in one workflow.

Incorporating human approval for sensitive operations.

Making operational decisions and execution results traceable.

Establishing a foundation for measurable reliability, safety, and workflow performance.

What we learned

Through this project, we learned that building useful AI agents requires much more than integrating a language model. Reliable systems need structured outputs, carefully defined tools, persistent state, clear permissions, error handling, and measurable evaluation.

We also learned the importance of separating probabilistic AI reasoning from deterministic business rules. Human oversight, evidence traceability, and post-action verification are essential when AI systems interact with operational workflows.

Most importantly, we learned to evaluate an agent by what it successfully verifies and accomplishes—not simply by the quality of its generated responses.

What's next for OpsPilot — The Evidence-Driven AI Operations Agent

Our next goal is to evolve OpsPilot into a more robust and adaptable operations automation platform.

Planned improvements include:

Integrating with real ticketing systems, monitoring platforms, and enterprise communication tools.

Expanding support for operational use cases beyond incident management.

Improving retrieval accuracy and evidence attribution.

Adding durable workflow recovery, idempotent tool execution, and more advanced failure handling.

Developing comprehensive evaluations for task success, evidence correctness, and unsafe-action prevention.

Introducing role-based access control, stronger audit protections, and deployment security.

Exploring adaptive workflows that improve from verified outcomes and human feedback.

Our long-term vision is to build an AI operations partner that can safely coordinate complex tasks, explain its decisions, verify its actions, and earn trust through demonstrable results.