opendevops

AI DevOps for businesses supported by Craftech, designed to resolve AWS issues faster. It investigates WhatsApp reports, applies engineer-approved fixes and verifies recovery.

Comment

Inspiration

When a company's orders stop processing, its team needs help restoring the service and knowing what is happening. Craftech manages cloud infrastructure for businesses facing these kinds of problems. Resolving them involves gathering technical context, investigating the cause, agreeing on a change and confirming that the service works again.

We examined 543 Craftech DevOps and cloud tickets across 62 projects. For 72 tickets classified as incidents or observability work, the median time from creation to closure was 12.2 days. Among the 69 with recorded engineering effort, the median was three hours. These historical records, categorized with AI assistance, include administrative closure delays and do not measure outage duration.

That gap shaped our approach: help engineers investigate sooner, make proposed changes easier to review, verify the outcome and keep the client informed throughout the case. We designed opendevops to shorten the path from a reported problem to a verified resolution.

We built opendevops to help shorten that process. A representative of the client company can report an issue through WhatsApp. The agent investigates within the company's AWS account and prepares a proposed fix for a DevOps engineer to review. After approval, it applies the change, verifies recovery and updates the client.

Our goal is faster resolution with less coordination work, while engineers retain control over infrastructure changes. Measuring that improvement in customer pilots is our next step.

What it does

opendevops is an AI DevOps agent that companies deploy in their own AWS accounts to investigate infrastructure problems and carry out approved fixes. Their own DevOps team can operate it independently, or work with Craftech. Each company controls its installation, permissions, operating budget and case history.

The agent uses Amazon Bedrock for AI, Amazon Bedrock AgentCore for its case and background-job engines, and AWS storage for its operational state. AWS Lambda transports runtime requests and runs a separate executor for approved infrastructure changes. AWS service usage is charged to the company's account. After deployment and channel setup, day-to-day operation does not require Craftech to host the application or approve its changes.

A representative of the business can report an issue through WhatsApp, such as “our orders are stuck.” The agent turns the request into a case, investigates the available evidence and prepares an exact proposed change with a rollback plan. Amazon SES notifies the DevOps engineer, who signs into the portal to review and approve it. A separate executor applies the authorized change; the agent verifies the outcome and updates the business through WhatsApp, using AWS End User Messaging Social. Engineers can also start cases through the portal.

Our demo follows a synthetic order stuck in a real AWS workload. The agent identifies a disabled connection between an SQS queue and its Lambda worker. After approval, it restores that connection and verifies that the same order has been processed. Actual AWS operations and delivered messages support the complete flow.

The agent also builds knowledge of the infrastructure it observes: a resource map, relationships and reusable diagnostic checks validated against observations. Pulse runs those checks periodically and can open a case when findings warrant investigation. It shows its current, partial coverage so engineers can see what has been checked and what remains unknown.

The goal is faster resolution and less coordination work, with the company retaining control of both the infrastructure and the agent that operates on it.

How we built it

We built opendevops as a serverless application that companies can deploy in their own AWS accounts using versioned CloudFormation packages generated from AWS CDK. Each installation has its own resources and configuration, so the company's DevOps team can operate it directly.

The Python agent uses Strands Agents SDK with models accessed through Amazon Bedrock. Two separate ARM64 Amazon Bedrock AgentCore runtimes run case investigations and background jobs such as chat, architecture discovery, Pulse, health and alerts. Lambda functions transport requests to those runtimes. The case runtime can call the separate approved-change Lambda executor; the jobs role cannot invoke it. DynamoDB preserves case state, approvals and budget records; S3 stores evidence and configuration. EventBridge schedules infrastructure inspections and Pulse checks. A React and TypeScript portal, served through CloudFront and S3, gives engineers access through API Gateway and Cognito.

Both AgentCore runtimes are deployed. A fresh Orders run verified exact approval, completion of the same order, a validated case and delivered WhatsApp closure, with correlated runtime events. It used one explicit Continue investigation after a partial report. Fresh chat also passed; earlier discovery and S3 checks retain their recorded scope. The published Orders video records the complete workflow before this runtime migration.

We separated the agent's reasoning from its authority to change infrastructure. Approval is bound to an exact proposal and rollback plan. Before writing, a separate executor checks the engineer's authorization, the permitted resource and whether its configuration has changed since investigation. The agent then gathers fresh evidence to verify the outcome. Budget and execution limits are enforced in code.

AWS End User Messaging Social connects WhatsApp conversations and case updates to this workflow. Amazon SES delivers approval requests and closure reports. Both services are called by the running application, with delivery evidence recorded alongside the case.

The demo adds an isolated SQS queue, Lambda worker and DynamoDB order record. That lets us verify a business outcome: the same order reported as stuck is actually processed after an approved repair.

Challenges we ran into

We wanted an agent that could learn a company's infrastructure and watch over it day and night, reducing the need for engineers to repeat the same discovery work. Our goal was an ongoing cycle: observe the environment, retain useful knowledge, detect problems, propose fixes, verify approved changes and keep that knowledge current as the infrastructure evolves.

Turning that goal into a working system meant generating reusable diagnostic checks from real observations and validating them before use. Pulse runs those checks automatically on a configurable schedule, hourly by default, and can open a case when evidence warrants investigation. Scheduled inspections refresh the infrastructure map and its associated checks. Coverage, unknown results and budget limits remain visible, and infrastructure changes still require DevOps approval.

The hardest part was making an AI agent useful around infrastructure while keeping permission to change it under human control. A diagnosis can be plausible and still be wrong. We bound approval to the exact proposed change and rollback, then made a separate executor check authorization, resource scope and configuration drift before acting.

Cloud operations also do not always return a clear result. During the Orders demo, the first change returned an uncertain response. The agent inspected the actual state, established that the change had taken effect and continued verification without repeating the write. It then checked that the original pending order had been processed before closing the case.

Another challenge was keeping the business representative and the DevOps engineer connected to the same investigation. WhatsApp, email and the portal needed to preserve case context across asynchronous steps, including the wait for approval. We persisted case state and notification records so the engineer could review the evidence and the business could receive the outcome.

Accomplishments that we're proud of

Our biggest achievement was completing a recovery that a business could actually verify. In our demonstration, an order was stuck because the service responsible for processing it was disconnected from its queue. A WhatsApp report started the investigation. The agent identified the problem, requested DevOps approval and restored processing after the proposed change was approved. The original order then completed, and a WhatsApp message confirmed the outcome.

The order completed about 42 seconds after approval. From receiving the WhatsApp request to recording the case report, the workflow took about 5 minutes and 40 seconds, including the approval wait. We checked the result in both the application and its worker logs.

We built this as a repeatable test on real AWS infrastructure, using synthetic orders to protect customer data and operations. We are proud of bringing investigation, engineering approval, recovery verification and customer communication together in a working system that a company can operate within its own AWS account.

What we learned

We learned to measure the whole support process carefully. Engineering effort, time waiting for approval and time until a ticket is formally closed describe different parts of the work. Our Jira research helped us identify opportunities, while the live tests showed which steps we could measure directly. A credible claim of faster resolution will require comparing equivalent cases.

We also learned to define success from the business's perspective. An AWS API call can succeed while the original problem remains. In the Orders scenario, recovery meant confirming that the specific order had been processed. That requirement changed the evidence the agent needed before closing the case.

We learned that autonomy needs clear boundaries when an agent can change infrastructure. The agent can investigate independently, but a DevOps engineer must review and approve the exact proposed change and its rollback plan before execution. This gives the engineer an opportunity to challenge the diagnosis, assess the risk and understand how the configuration could be restored if the change fails. Human approval and a proposed rollback are essential parts of how we designed the system.

Finally, useful infrastructure knowledge needs maintenance. Resources change, observations become stale and checks cover only part of an environment. The agent needs to refresh what it knows, retain the evidence behind its conclusions and make uncertainty visible to the engineer reviewing its work.

What's next for opendevops

Our first priority is to test opendevops more extensively with companies and their DevOps teams, then improve it through their feedback and the outcomes of real cases. We want to measure resolution time, engineering effort, approval quality, rollback outcomes and operating cost across comparable scenarios, using our Jira research as a starting point.

We also plan to address more of the work found in those tickets, including scheduled maintenance and planned infrastructure changes. This will require controlled access to GitHub or the company's infrastructure repositories, so the agent can understand the infrastructure code, propose changes for review and work through the team's existing deployment process. Human approval and a proposed rollback will remain part of that workflow.

Finally, we want to make opendevops easier to install and use, with clearer onboarding and simpler configuration of permissions, monitoring and communication channels. We plan to prepare it for AWS Marketplace distribution while preserving deployment in the company's AWS account and operation by its own DevOps team.

Built With

Share this project:

Updates

Submission history