SiteKeeper

A small business writes "our Saturday hours are now 8 to 2" in a GitLab issue. Duo agents ship it - tested, scanned, deployed, health-checked and rolled back if it breaks.

Comment

Inspiration

Most small-business websites are not maintained by anyone with a pipeline. They are kept by one-person web agencies juggling dozens of clients, and every "please change our phone number" email is ten minutes of hand work that goes wrong in the same ways every time.

We know the ways because we measured them. In October 2026 we scanned 29,412 real local-business websites (27,784 answered). The defects that keep coming back are small and costly: images with no alt text (32% of sites), no meta description (20%), a phone number you cannot tap on a phone (14%), dead links (6%), no mobile viewport (4%). sitecheck tests for those, plus the ones a hand edit at 6pm introduces: a phone number on the page that differs from the one in the structured data Google reads, broken JSON-LD, mixed http content and unlabelled form fields.

So the question was: what if the whole life of a website change after the request - the edit, the review, the tests, the security scan, the deploy, the check that it is really live, and the rollback when it is not - ran without anyone touching it?

What it does

The owner writes an issue in plain English: "Our new phone number is 217-555-0199."

  1. Duo flow ship, triage. An agent reads the issue and the site, decides whether it is in scope, finds every file that holds the fact, and posts the plan on the issue in plain English. Its answer opens with one fixed line (SCOPE: ...), and the build agent's first instruction is to stop on SCOPE: out_of_scope, so requests like "redesign the logo" get a polite note and stop there.
  2. Duo flow ship, build. A second agent branches, edits every copy of the fact (the visible number, the tel: link, the JSON-LD), runs the defect checker and the unit tests locally, pushes, and opens a merge request that closes the issue.
  3. Merge request pipeline. Unit tests; sitecheck reported as GitLab Code Quality and a JUnit test report, so the defects show inline on the MR; Secret Detection; SAST; build; and preview-verify, which serves the exact build artifact inside CI and smoke-tests every page before anything reaches production.
  4. Hands-off merge. The flow's own token cannot arm a merge (GitLab refuses it with 403 insufficient_scope, and no Duo tool sets "merge when pipeline succeeds"), so the MR pipeline does it: an arm-merge job on the flows' sitekeeper/* branches arms the merge with a token held as a masked CI variable, pinned to the commit that pipeline is testing. The MR merges the moment it is green. Owners who want to approve each change leave the token out, and the MR waits for one "merge when pipeline succeeds" click.
  5. Main pipeline. Deploys to GitLab Pages as the production environment, then live-health waits until Pages is serving this commit (every page carries its build version), proves every page answers 200 and still has a tappable phone link. If the live check fails, rollback republishes the previous commit's site.
  6. Duo flow fix. On any failed pipeline an agent reads the failing jobs and their logs, classifies the cause (a site defect, a committed secret, a SAST finding, a failing unit test, or infrastructure - which ends the flow rather than "fixing" a runner outage in code), fixes it on a branch, pushes, opens the MR (armed to merge the same way), and explains what happened on the MR and the issue.
  7. Daily schedule. monitor-live re-checks the live site every day, so a site that breaks for reasons outside the repo still opens a failed pipeline - and the fix flow sees it.

Post-code stages automated: review of the change (the checker plus the agent's plan), testing, secret detection, SAST, pre-production verification, deployment, post-deploy health checking, automatic rollback, failure diagnosis and repair, scheduled monitoring, and a summary back to the owner on their issue.

Two real runs on gitlab.com, 2026-10-15 (UTC):

  • Issue #3 "We are open Saturdays now, 8am to 2pm", assigned to the ship flow, zero clicks: issue opened 18:54:41, plan posted 18:55:50, commit 18:56:12, MR !2 opened 18:56:17 (4 pages: the footer text and the JSON-LD openingHours), arm-merge armed the merge, MR pipeline green and merged 18:57:28 (issue closed by "Closes #3"), main pipeline 2952402957 deployed and live-health verified the new build at 18:59:46. Total: 5 min 5 s, no human act after the issue.
  • Issue #2 "Our new phone number is 217-555-0199", before arm-merge existed: plan 16:14:56, MR !1 16:15:22 (16 lines: text, tel: links, meta descriptions, JSON-LD), merged 16:17:11 after one "merge when pipeline succeeds" click, live-verified 16:19:01. That run is why arm-merge exists.

How we built it

  • checks/sitecheck.mjs - the defect checker, zero dependencies, built from the 29,412-site scan's most common defects. It reports as text, JSON, JUnit and GitLab Code Quality, so the same check is a local command for the agent, a test report and an inline MR annotation. Tested with node --test.
  • .gitlab-ci.yml - the pipeline above: templates for Secret Detection and SAST, merge-request and default-branch rules, a production environment, Pages deploy, a live-health job that polls for its own commit, a rollback job, and a scheduled monitor job. Validated against GitLab's own CI schema before import.
  • .gitlab/duo/flows/sitekeeper-ship.yml and sitekeeper-fix.yml - two Duo Agent Platform flows, two agents each, registered in the AI Catalog. The first agent (triage, diagnose) must open its answer with one fixed line - SCOPE: website_change or SCOPE: out_of_scope, CAUSE: one of five - and the second agent's first instruction is to read that line and stop on out_of_scope or infra. The second agent's tools are only the ones its job needs (read, edit, run the checker, git, open an MR, note). The flows run on the default flow image (git, Node), so no extra agent config is needed.
  • scripts/ - build (stamps the commit into every page), serve, and smoke, the same scripts in CI and on a laptop.
  • site/ - a fictional plumber's website in plain HTML: the kind of site this is for.

Challenges we ran into

  • Proving "live" means this commit. Pages publishes asynchronously, so a 200 right after deploy can be the old site. Every page now carries its build version and live-health waits until the live site serves it before checking anything.
  • Staging on the Free tier. Pages parallel deploys need Premium, so staging is preview-verify: the exact artifact, served the way Pages serves it, inside CI.
  • Routing an agent's decision safely. Our first flows used response schemas and conditional routers; the AI Catalog rejected them ("Versions is invalid"). So routing is unconditional and the decision travels as a fixed first line that the next agent must check before it acts. If an agent ever misreads it, the change still has to pass the same tests, scans and preview as any other MR before it can merge.
  • Rollback without rewriting history. Rollback republishes the previous site and leaves the commit in place; the fix flow opens the forward fix, so the history stays honest.

Accomplishments that we're proud of

  • An owner's sentence becomes a verified live change in about five minutes with no human step after the issue, and a broken change undoes itself.
  • The checker comes from real data (a scan of 29,412 sites), not a guess at what breaks.
  • The same pipeline catches a committed secret, and the fix flow removes it and tells the owner it still has to be rotated - the one step no agent should claim.

What we learned

The agent is the easy part to demo and the hard part to trust. Trust came from the pipeline around it: every agent output goes through the same tests, scans, preview and live check a careful human's would, and the agent's decisions are typed so they cannot route themselves somewhere unexpected.

What's next

  • One project per client site, with the agency's own checks added to sitecheck.
  • Multi-site monitoring: one schedule, many sites, one dashboard of issues.
  • More fact types: menus, prices, staff pages, holiday closures.

AI tools used

The code, pipeline, flows and this text were written with Anthropic's Claude as a coding assistant; the agents inside the product are GitLab Duo Agent Platform flows.

Built With

Share this project:

Updates

Submission history