AI Incident Commander

AI-powered incident management platform that persistently stores historical knowledge and generates short- and long-term fixes using semantic memory. Shortens incident response times.

Comment

Inspiration

Modern engineering teams rely on AI to accelerate incident response, but many solutions lack persistent memory of previous incidents. We wanted to build an AI-powered Incident Commander that not only analyzes live production incidents, but also learns from historical incidents using semantic search. By combining Amazon Bedrock with CockroachDB's distributed vector indexing, the application can retrieve similar incidents from long-term memory and provide context-aware recommendations during an outage.

What it does

AI Incident Commander helps Site Reliability Engineers (SREs) respond to production incidents faster.

Users can:

  • Create and manage production incidents
  • Automatically generate AI summaries using Amazon Bedrock
  • Store semantic embeddings in CockroachDB
  • Search historical incidents using vector similarity search
  • Ask an AI Incident Commander to investigate new incidents using previous operational knowledge
  • View incidents through a lightweight web dashboard

Instead of answering in isolation, the AI retrieves relevant historical incidents before generating its recommendations.

How we built it

The backend was built with FastAPI and Python.

Each incident is stored in CockroachDB Cloud. After creation:

  • Amazon Bedrock generates an incident summary.
  • Bedrock generates a vector embedding.
  • The embedding is stored inside CockroachDB using Distributed Vector Indexing.
  • When an engineer asks a question, the system performs semantic similarity search over historical incidents.
  • Those results become context for the AI Incident Commander, which generates root cause analysis and recommended remediation steps.

The frontend is a lightweight HTML, CSS and JavaScript dashboard that communicates with the FastAPI backend.

The application was deployed on Render for public access.

Challenges we ran into

The biggest challenges were integrating vector search with CockroachDB and deploying the application.

We worked through several issues involving vector formatting, embedding storage, SSL certificate configuration for CockroachDB Cloud, and deployment environment differences between local development and production.

Ensuring that semantic search, AI analysis and persistent memory all worked together reliably required careful debugging across multiple services.

Accomplishments that we're proud of

  • Built a complete AI-powered incident management system in a hackathon timeframe.
  • Successfully integrated Amazon Bedrock for summarisation, embeddings and reasoning.
  • Implemented persistent semantic memory using CockroachDB Distributed Vector Indexing.
  • Deployed a fully functional public application.
  • Created an AI workflow that retrieves historical incidents before generating recommendations instead of relying only on the language model.

What we learned

This project provided hands-on experience building retrieval-augmented AI applications.

We learned how vector databases improve AI reasoning, how persistent memory changes agent behavior, and how to integrate cloud AI services with production databases.

We also gained experience deploying AI applications that combine databases, cloud APIs and web frontends into a single production system.

What's next for AI Incident Commander

Future improvements include:

  • Multi-user authentication and RBAC
  • Slack and Microsoft Teams integrations
  • Automatic log and metrics ingestion
  • Bedrock Agents for autonomous investigation workflows
  • Knowledge Base integration for operational runbooks
  • Real-time monitoring dashboards
  • Incident timeline generation
  • Root cause clustering across historical incidents

Built With

Share this project:

Updates

Submission history