Lifeguard Agent

An AI on-call engineer for small teams. It runs on three NVIDIA Nemotron models that find why the server broke, and nothing is fixed until a human gives approval.

Comment

Inspiration

During my IT operations internship at Sears, I did night and weekend support. When something broke, finding the cause was on me, and for the big outages I had to call the on-call manager. Small start-ups and student teams usually have nobody to call. The goal of Lifeguard-Agent is to be that extra pair of hands. It does the first investigation, explains what broke and why, and suggests a safe fix that only runs after a human approves it.

What it does

Lifeguard-Agent is an AI on-call engineer for small teams that run their app on one Docker server. You break a demo shop, and the agent investigates it like an SRE would.

  1. Nemotron Nano reads the alert and guesses where to look.
  2. My code checks a memory of past incidents for a close match.
  3. Nemotron Super investigates with read-only tools like docker ps, docker logs and df -h, and searches the web once with Tavily.
  4. Nemotron Ultra checks Super's report and gives the root cause and up to 3 fix options, each with its risks and a way to undo it.
  5. A safety check in my code blocks any option that could lose data or can't work.
  6. You approve on the page or with two taps in Telegram. Nothing changes without that.
  7. The fix runs, and it only counts as FIXED when the customers' own action works again, like placing an order.
  8. A short postmortem is written for you.

You can also type your own problem and try to stump the agent. After you decide, the page shows the hidden answer next to the agent's answer.

The same agent also fixed 3 real faults on real Docker containers on a Nebius AI Cloud VM.

How I built it

  • Three NVIDIA Nemotron 3 models on Nebius Token Factory. Nano does the quick first look, Super runs the long investigation with tools, and Ultra makes the final call.
  • I tested this instead of guessing. In two rounds on 20 faults, Super alone found causes as well as the team. But Ultra's fix options were safer, so Ultra decides what a human approves.
  • Tavily for one web search per incident. My code checks every source link against the real search results.
  • Python and FastAPI on the backend, with the live timeline streamed to the page.
  • A simulated server for the public demo, and the same tools on real docker commands for the real server.
  • Safety rules in code, a short menu of 5 fix actions, and spending limits on the demo.
  • A plain HTML, CSS and JavaScript page with a 3D view of the shop (three.js), and a Telegram bot as an on-call pager.
  • Docker on Render with tests on GitHub Actions, my own domain on Cloudflare, and uptime monitoring.
  • 126 automated tests, a load test with 50 visitors at once, and the 2-round evaluation.
  • I built it with AI-assisted development, using Claude Code as my coding assistant. The idea and the design decisions are mine, and I tested and deployed every version myself.

Challenges I ran into

  • The biggest model suggested something dangerous. For a full PostgreSQL disk, Nemotron Ultra said to delete old WAL files, which can corrupt a database. Since then, fixes only come from a short list of safe actions, my code checks each one, and nothing runs without a human.
  • Models can be out of date. Super suggested a PostgreSQL setting that was renamed years ago. That's why the agent searches the web and shows its sources.
  • Making visitors' own problems fair. Sometimes Ultra built faults that couldn't be fixed or gave the answer away. Now my code tests every fault and tries to fix it first. If my code can't fix it, the agent isn't asked to.
  • The numbers surprised me. I expected three models to beat Super alone, and they didn't. Super found causes just as often for half the cost. I wrote that in the README honestly and kept Ultra for the final call, because its fixes were safer.

Accomplishments that we're proud of

  • It's a real product on my own domain, lifeguard-agent.com, not just slides. It's the first time I bought a domain and set up the hosting and monitoring myself.
  • It fixed 3 of 3 real faults on a real server.
  • In 100 test runs it never got a cause completely wrong, and every fix that Super or Ultra recommended worked (80 of 80).
  • You can approve a fix from your phone with two taps.

What I learned

  • A prompt asks, code makes sure. Super skipped a rule in my prompt, so now my code does that step itself.
  • Test AI twice before you believe a result. The same test gave 2 more right answers the second time, just by luck.
  • Real cloud and DevOps work, like DNS, Docker, CI/CD, monitoring, a cloud VM and SSH.
  • How to read an incident in five questions. What do users see, what's broken, what's wrong, why did it happen, and how do I fix it safely?

What's next for Lifeguard Agent

I want Lifeguard-Agent to grow from small teams to real production systems at bigger companies.

  • Start by itself from real alerts, like UptimeRobot or Prometheus.
  • Handle more kinds of systems, like queues and background workers.
  • Support Kubernetes.
  • Smarter memory that finds past incidents by meaning, not just by words.
  • Team features like many servers, team approvals and a record of every action.

Built With

Share this project:

Updates

Submission history