NightShift — an autonomous AI ops fleet for Shopify

A multi-agent AI workforce that runs your Shopify store overnight — it detects, reasons about, safely fixes, and verifies operational failures before you wake up, and hands you one paragraph.

Comment

The problem

Every Shopify merchant goes to sleep a little worried. A theme deploy silently drops the "Add to Cart" button. A new discount code stacks with an old one and sells everything at 90% off. A tracking pixel disappears and a week of attribution data is gone. You find out from the next morning's sales report — hours too late, thousands of dollars down.

A chatbot doesn't help. It can't watch your store, and it can't fix anything.

What NightShift is

NightShift is an AI employee that does both. Five specialist agents run an orchestrated "shift" every night against your live store:

Agent Watches Acts
Product Quality Catalog Generates ALT text (auto); rewrites thin descriptions (approval)
Checkout Specialist Active discounts Deactivates redundant / overlapping stackable codes
Theme Guardian A watch-list of critical theme files Restores where Shopify grants the write; else an honest diff
Tracking Specialist Installed script tags Recreates a vanished tracking pixel from its own snapshot
Chief Ops AI Every specialist's findings One Gemini call → an executive briefing with a stated root cause

Every finding — whichever agent catches it — moves through the same eight-step lifecycle: Observe → Reason (Gemini) → Assess Risk → Approve → Execute → Verify → Explain → Persist. A fix is auto-applied only when its risk classification says it is genuinely safe and reversible; anything customer-facing waits in the Approval Center. Every executed fix is re-verified with a read-after-write against Shopify's live data, and rolled back if it didn't take.

How we built it

  • Orchestration — a Celery DAG (inspect_catalog → inspect_discounts → inspect_theme_files → inspect_tracking_scripts → plan_cognitive_tasks → compile_shift_report) on Cloud Run, triggered by Cloud Scheduler. The same fleet is also expressed as a Google ADK SequentialAgent of five LlmAgents, loadable with adk web.
  • Reasoning — every specialist's analysis and Chief Ops AI's cross-agent synthesis go through Gemini 3.6 Flash on Vertex AI (billed through the project, not an API key).
  • Security / governance — every agent runs under a named AgentPrincipal with a least-privilege GCP service account and an explicit Shopify-mutation allow-list. Every agent-initiated mutation passes through one Agent Gateway chokepoint; a disallowed action becomes an AGENT_SCOPE_VIOLATION and a GATEWAY_DENY audit row, never a silent proceed. Every Gemini prompt and response is screened by Vertex AI Model Armor — a block degrades to a deterministic path and is recorded as AI_INPUT_BLOCKED / AI_OUTPUT_BLOCKED.
  • Durable state — Cloud SQL (Postgres 16, Postgres RLS per store) holds missions, issues, immutable shift reports, an append-only audit_logs table, and a Memory Bank of learned merchant preferences that is re-read at the next plan step days later. Idempotency keys stop a retried task from double-mutating Shopify. Memorystore Redis backs the Celery broker and a daily LLM budget guard that survives worker restarts.
  • Telemetry — OpenTelemetry spans over the whole reasoning chain, exported to Cloud Trace (CloudTraceSpanExporter), with queue-hop context propagation so plan → execute → verify → report land in one trace tree.
  • Everything else — Secret Manager for every credential, Cloud Build + Artifact Registry for the images, an embedded Next.js admin UI on Vercel, 397 hermetic backend tests (pytest, in-memory fakes for every port, no key / no account / no network).

What we learned

  • Cloud Run's default is CPU-only-during-requests — a background Celery worker silently freezes between health probes until you set --no-cpu-throttling. That single flag was the difference between "it deploys" and "it actually processes a shift."
  • Model Armor is genuinely easy to bolt on as a decorator around a structured LLM client — the hard part is making every downstream caller degrade gracefully when it blocks, which fell out for free because a ModelArmorBlockedError subclasses the existing schema-validation error every caller already handles.
  • "Success is a contract, not a status code" — verifying the outcome (read-after-write against the real API) rather than "did the step run" is what makes autonomous mutation trustworthy.

Challenges

  • Getting a Celery worker to behave on Cloud Run (CPU throttling, a health-shim for the required $PORT listener, a Serverless VPC connector to reach Memorystore).
  • A Windows core.autocrlf checkout shipped a #!/bin/sh\r shebang that made Cloud Run fail with "no such file or directory" — fixed with .gitattributes + a build-time sed.
  • Keeping the in-code fleet and the registry / ADK graph in lockstep so GET /api/v1/agents, the Operations Center panel, and adk web all show the same five versioned agents.

Try it

Built With

Share this project:

Updates