SentryGrid

A Gemini-powered multi-agent system on Google Cloud for unattended machine operations: safe recovery, human-approved outreach, technician field plans, and outcomes verified by live machine data.

Comment

Inspiration

49 unattended self-serve coffee machines are supported by only three part-time technicians. Their most valuable work is physical: repairing machines, maintaining equipment, and solving problems on site. Yet based on our current workflow estimate, roughly 50 technician-hours each month are spent checking machine status, calling site hosts, and planning visits. We built SentryGrid to protect that limited human attention. The fleet handles routine digital work. Technicians focus on what only humans can do.

What it does

SentryGrid is a Gemini-powered multi-agent operations system for unattended machine fleets. It helps the fleet:

  • Recover a machine remotely when policy allows
  • Prepare site-host outreach but require a technician to approve it
  • Turn physical exceptions into prioritized field plans for technician review
  • Use verified, machine-specific history to reduce unnecessary alerts
  • Keep every incident open until a fresh machine reading confirms the result SentryGrid runs asynchronously from machine events, delegates work to specialized roles, calls tightly scoped tools, updates operational state, and verifies the physical result.

Built for the Unlikely Heroes

The heroes of SentryGrid are not office workers with time to supervise an AI assistant. They are three part-time technicians responsible for a city-wide fleet.

SentryGrid is designed around their reality:

  • Routine digital work should happen in the background
  • Outward communication should remain under human control
  • Physical work should arrive with the reason, history, supplies, and priority already prepared
  • A technician should not be sent merely because an API returned an error

Five Roles, Five Narrow Jobs

SentryGrid separates reasoning, communication, planning, supply analysis, and machine control.

  • SHARP EYE — Incident coordinator agent: Checks current machine state, retrieves relevant history, selects the next safe path, and delegates work.
  • ALEX — Supply intelligence agent: Interprets bean and water readings, recent usage, and verified machine history.
  • HERALD — Communication agent: Drafts context-aware messages for site hosts. It cannot send them without technician approval.
  • PLANAGER — Field-planning agent: Converts machine condition, history, location, and supply needs into a prioritized daily plan for technician review.
  • BOLT — Deterministic recovery worker: Executes one scoped restart through the Machine Control API. BOLT is deliberately not an open-ended agent and has no model loop.

That separation matters. A communication agent cannot restart a machine. A planning agent cannot assign a technician automatically. A recovery worker cannot keep retrying until something happens.

Three Governed Outcomes

1. Safe Remote Recovery

When machine A9 stops reporting, SHARP EYE first checks whether the machine is actually offline and whether recovery is allowed. It then delegates one bounded action to BOLT. BOLT sends one scoped restart through the Machine Control API. Duplicate commands are rejected through idempotency controls, and BOLT cannot enter an uncontrolled retry loop. A successful command response does not close the incident. SentryGrid waits until A9 sends a fresh status reading that confirms it is online again. The command receipt proves that a restart was accepted. The machine reading proves that recovery happened.

2. Human-Approved Host Outreach

When MANGO01 is online but low on beans, restarting would solve nothing. ALEX confirms the supply problem. HERALD prepares a refill request in Mongolian for the site host. A technician reviews the draft and decides whether to send it. Even after the message is sent, the incident stays open. The site host refills the machine, and SHARP EYE waits for a new sensor reading to confirm that the supply level has recovered. A message receipt proves that communication happened. A fresh machine reading proves that the problem was solved.

3. Technician-Ready Field Planning

Some problems need human hands. For those cases, PLANAGER prepares Today's Plan: a prioritized list of field visits for the technicians to review. PLANAGER prepares the work; it does not dispatch anyone. The technicians review the plan, choose the owner, and keep control of the physical workflow. The planning path is also governed. If a candidate response violates the required plan schema, the validator rejects it. PLANAGER gets one bounded correction attempt before the plan can be shown for review.

Machine-Scoped Memory, Not One Fleet-Wide Rule

Different machines behave differently. Applying one threshold to all 49 machines creates noise and wastes technician attention.

ALEX uses each machine's verified history as machine-scoped context. For MANGO01, it adjusted the low-water alert threshold from 35,000 mL to 20,000 mL, reducing false alerts without changing the rule for the rest of the fleet.

Cloud Firestore remains the authoritative record of machine state and incident history. Agent Platform Memory Bank carries derived, machine-scoped context across incidents. Memory can inform a decision, but it cannot replace fresh evidence from the machine.

How we built it

SentryGrid follows a closed, event-driven control loop:

  1. Observe: The existing Friday machine telemetry API receives live machine and sensor readings.
  2. Store: Cloud Firestore holds authoritative machine state, telemetry, and incident records.
  3. Transport: Relevant state changes are published through Pub/Sub for durable, decoupled delivery.
  4. Start: An authenticated Cloud Run dispatcher creates one idempotent incident.
  5. Coordinate: SHARP EYE reasons with Gemini 3.5 Flash through Google ADK and delegates only the work required.
  6. Govern and act: The system chooses one of three paths—bounded remote recovery, human-approved host outreach, or a technician-ready field plan.
  7. Verify: SentryGrid waits for a fresh reading sent by the physical machine.
  8. Close: Firestore is updated, the incident correlation record is completed, and SHARP EYE closes the incident.

Event → decision → delegation → action → machine evidence.

That final return path is the core of SentryGrid. Software does not declare success on behalf of the physical world.

Built on Gemini and Google Cloud

SentryGrid uses Gemini 3.5 ** through the **Google Agent Development Kit (ADK).

The deployed architecture includes:

  • Gemini Enterprise Agent Platform for managed agent operations
  • Agent Registry for registered and versioned specialist roles
  • Agent Runtime for managed execution
  • Agent Platform Memory Bank for persistent, machine-scoped context
  • Agent Identity and Agent Gateway for caller, route, and tool-scope controls
  • Model Armor for screening untrusted Fleet Chat prompts and responses
  • Cloud Run for the authenticated incident dispatcher
  • Pub/Sub for durable event delivery
  • Cloud Firestore for authoritative machine and incident state
  • Cloud Logging for operational evidence and incident correlation Every incident carries one correlation ID from the SentryGrid interface into Cloud Logging. That ID connects the original event, SHARP EYE's decision, the delegated action, and the final machine reading. ## Security and Governance

Autonomy is useful only when its boundaries are clear.

SentryGrid allows the system to observe machine state, retrieve verified history, coordinate specialists, perform one policy-approved restart, verify the result, and close the incident.

It requires human approval before communicating with a site host. Field plans are prepared for review, not automatically assigned. Incidents fail closed if fresh machine evidence does not arrive.

For untrusted Fleet Chat input, the security path is:

Fleet Chat → Agent Identity and Gateway → Model Armor → SHARP EYE

In our demo, a malicious request tries to override instructions, reveal credentials, mark a machine as verified without evidence, and restart all 49 machines without approval.

Model Armor blocks it before agent processing, producing:

  • Zero agent calls
  • Zero memory writes
  • Zero tool calls

A normal question about a machine's water level is allowed to continue.

Additional controls include:

  • Machine- and incident-scoped tool permissions
  • One-shot actuation
  • Idempotent incident creation and restart commands
  • Bounded waits and bounded correction attempts
  • Human approval for outward communication
  • Schema validation before field plans reach technicians
  • Closure only after fresh machine evidence
  • Correlation IDs across the UI and Cloud Logging

Proof That It Works

Our demo shows the system acting, not merely describing an action:

  • A real coffee machine goes offline, receives one restart, boots, and reports itself online again
  • A low-supply incident produces a Mongolian refill draft that requires technician approval
  • A new supply reading closes the refill incident
  • ALEX applies machine-scoped history to reduce false low-water alerts
  • PLANAGER produces a prioritized daily field plan and corrects a schema-invalid candidate through a bounded validation path
  • Model Armor blocks a malicious fleet-wide request before any agent, memory, or tool call occurs
  • The same incident ID appears in the SentryGrid interface and Cloud Logging
  • The Cloud Run service page provides visual proof of the deployed backend and active revision

The repository also includes a seeded fleet and a mock hardware adapter so the workflows can be reproduced without access to Enbotics' proprietary physical machines.

Challenges we ran into

The hardest problem was defining what finished means in the physical world. A successful API response only proves that a command was accepted. It does not prove that the machine recovered, that beans were added, or that an alert disappeared. We therefore kept incidents open until the machine itself reported the expected state.

We also had to:

  • Prevent recovery from becoming an uncontrolled retry loop
  • Separate safe internal actions from outward communication
  • Give every role a narrow responsibility and tool boundary
  • Make several specialists understandable in one interface
  • Connect asynchronous software decisions to real physical equipment
  • Build the complete prototype in roughly ten days

Accomplishments that we're proud of

  • Connected the workflow to a real physical coffee machine
  • Built machine-verified incident closure
  • Created a deterministic, one-shot recovery worker
  • Added human approval before host communication
  • Built a daily field-plan interface
  • Demonstrated Model Armor blocking unsafe input before agent processing
  • Integrated an Ops Chat interface where agents dynamically explain policy adjustments based on historical consumption data.
  • Connected the user-facing incident to Cloud Logging with one correlation ID
  • Used Pub/Sub, Cloud Run, and Firestore for event-driven execution and state

What we learned

Useful autonomy is not about removing people from every decision. It is about deciding:

  • What software can handle safely
  • Where human approval is necessary
  • What evidence is required before declaring success
  • What context people need when physical work remains Our technicians are not a fallback. They are the unlikely heroes SentryGrid is designed around.

What's next for SentryGrid

Coffee is our proving ground. The same operating model can support other unattended equipment fleets with live machine data, remote controls, and physical exceptions:

  • Self-service kiosks
  • EV charging stations
  • Micromobility fleets
  • Refrigeration equipment
  • Distributed water systems Next, we plan to expand machine coverage, improve route and supply planning, strengthen identity and tool permissions, and measure technician time reclaimed in production.

Prior Work and Hackathon Scope

Our physical coffee machines, fleet operations, and operational data sources existed before the hackathon. They provided the real-world problem, equipment and operating environment for this project. SentryGrid itself was created entirely during the hackathon submission period.

Built With

  • a2a
  • agentregistry
  • cloud-firestore
  • cloud-run
  • cloud-scheduler
  • cloud-tasks
  • eventarc
  • gemini
  • google-cloud
  • google-cloud-iam
  • googleadk
  • memorybank
  • modelarmor
  • nextjs
  • pub/sub
  • python
  • vertexai
Share this project:

Updates