Knowledge Base Assissted SRE Conversational Agent

An agentic approach to a Site Reliability Agent, adaptive to new information through a maintainable knowledge base of the current outage and investigations.

Comment

Inspiration

There is no doubt that LLMs can harness a great amount utility to engineers, be it SWE or SRE. Fixing bugs is not easy. Making SREs write up postmortems, keeping track of progress on multiple outages, and keeping stakeholders informed, takes away from their capability to focus on solving the outage.

The goal is to reduce this administrative and context-switching strain, whilst still providing full power to the on-call SREs to actually make final call.

What it does

A knowledge base is maintained mutually with the SRE and the agent. A language model is to interact with it through tool calling. This keeps the current state of the LLM up to speed with the outage, being able to suggest next debugging steps, as well as on-boarding incoming SREs to the outage.

We bring the agent right where the engineers are: Discord.

Type fast? Use text input. Prefer hand-free? Join the SRE agent in the voice channel.

How we built it & Challenges we ran into

The initial vision was: to have a very general interface for using diffs of any knowledge base to reason only about relevant steps through a web platform. As a solo entry, focusing time on a web platform, on top the architecture design, brainstorming, idea validation, would have been a significant time sink for little reward.

Early on a decision was made to pivot from a web platform approach, to using Discord as the UI. At this point the pivot to focus on SRE was made.

Accomplishments that we're proud of

The creation of a speech/conversational SRE agent with tool calling for RAG is no easy task. Especially as one person I am very proud of staying motivated to see this project to an end, creating an MVP.

What we learned

  • Don't be scared to pivot. Pivotting early helped this project significantly.
  • LLMs can fail in many more ways than some might expect; there should be a lot of caution about offloading full control to SRE agents in the current state.

What's next for Knowledge Base Assissted SRE Conversational Agent

  • GitHub API integrations
  • Completion of Speech to Text during calls with the SRE.

Built With

  • automated-speech-recognition
  • discord
  • fastapi
  • go
  • kokoro-tts
  • onnx
  • openai-whisper
  • openrouter
  • opus
  • rag
  • text-to-speech
  • uv-python
Share this project:

Updates