Foreman+

Inspiration

A field technician photographs a water heater's nameplate and tells a homeowner "about $4,000" — and that number never gets checked against anything. Six weeks later a different technician shows up for a callback with zero memory of the first visit: what was diagnosed, what was quoted, what the customer actually said. Every fact traveled through someone's head or a text thread, and nothing in the chain ever asked "how do we know that's true."

We wanted an agent fleet that treats "how do we know" as a first-class question — not by trusting the model more, but by making it defend every claim before it becomes shared truth. And we wanted the capture step to cost the technician nothing: no typing, no app to learn, just a photo and their normal voice.

What it does

A technician photographs the equipment nameplate and talks. Foreman+ turns that into a verified job record on the spot — a price they can quote before leaving the driveway, every number traceable to where it came from, and the next person on the job (a comfort advisor, a callback tech, a home-warranty authorizer) arriving with the property's memory instead of starting from zero.

Three things make that work. First, capture is zero-cost with an honest UNKNOWN: the intake agent reads the equipment model and serial number out of the image itself, not out of the technician's words, and when a plate is unreadable the field stays UNKNOWN rather than getting silently guessed. Second, one set of gate-verified facts renders for two audiences — a technician-facing stamp that survives a later dispute, and a homeowner-facing document that shows exactly which facts and flags produced the number in front of them. Third, property memory outlives the visit and the person: every fact, approved and rejected, sits in one shared store the whole fleet reads, so the closer agent can brief whoever shows up next on what's verified, what was rejected and why, and what similar jobs the fleet has seen before.

The whole thing runs two live seats: an office workspace (properties → property record → visit → ledger) and a technician seat that opens on a phone — briefing before the visit, photo + hold-to-talk after. It's also wired to Mentra Live smart glasses, so a technician can file a job hands-free without putting down a wrench; capture is device-agnostic, so the same intake contract will take the Android XR glasses shipping this fall without a fleet change.

How we built it

Three Google ADK agents (foreman, estimator, closer) share one gated Postgres memory store and deploy as independent Cloud Run services. foreman records reported attributes into memory and hands off via ADK's native sub_agents transfer; estimator searches the fleet's memory for similar past equipment/issues before writing back a scoped estimate; closer never writes memory, only reads it, and is the only agent exposed outside the fleet — over A2A — as its own Cloud Run service, giving any downstream FSM or home-warranty system a CRM-agnostic exit.

Nothing reaches shared memory by being said. Every proposed fact passes through a write-gate: cheap guards (registered fleet member, value size, predicate cap) run before any billed LLM call, then a gemini-3.7-flash verifier judges the proposal against the subject's existing facts — labeled explicitly as data to judge, never instructions, so a fact that tries to talk to the verifier is itself grounds for rejection. Verifier failure fails closed. Rejections are never deleted; they stay in the gate journal with their reason and surface in the closer's closeout document.

Reasoning runs on gemini-3.7-flash through Vertex AI for all three agents and the verifier. gemini-embedding-2 (768 dims, HNSW cosine index) embeds facts in Cloud SQL Postgres + pgvector for semantic recall across jobs. Firestore carries a best-effort live activity feed that powers the dashboard without ever being the system of record. Gemini Live (gemini-live-2.5-flash) drives a hands-free guidance brain for the glasses leg, holding the technician's latest camera frame as its "eyes." gemini-2.5-flash-image generated the sample nameplate photos seeding the demo workspace, and gemini-2.5-flash-tts renders spoken responses on the dashboard. Three more Google AI models run in supporting, honestly-labeled roles: Gemma 4 (gemma-4-26b-a4b-it on Vertex serverless MaaS) triages the office seat's open questions — a category + urgency label on every refused claim and unknown field, explicitly advisory, never writing memory; Veo 3.1 generated the field-scene clip on the office intro and the submission video's b-roll; Lyria (lyria-002) generated the optional ambient loop a judge can play while the demo fleet works. OpenTelemetry, via ADK's --otel_to_cloud, nests the write-gate's own span under ADK's agent/tool spans so one Cloud Trace waterfall shows intake → LLM → gate → database for a single request. foreman-glass (Bun + the Mentra SDK) bridges the glasses; foreman-brain runs the persistent Live API session; foreman-dash is a separate FastAPI service with no LLM calls of its own, deriving properties, briefings, and open questions from memory_facts + gate_journal on every request rather than a separate table.

Challenges

The write-gate's whole job is resisting the thing an LLM naturally does — agree. Getting a verifier that actually rejects a plausible-sounding claim against a contradicting nameplate fact, and does so fail-closed on its own errors, took several rounds of adversarial review before it held.

Vertex's Dynamic Shared Quota answered our seed script's back-to-back visits with 429s inside both the verifier and the agents' own model calls, surfacing as flat HTTP 500s from /run — fixed by giving every agent's model HttpRetryOptions and pushing the Cloud Run request timeout to 600s.

The glasses leg fought us hardest. The vendor's documented single-shutter photo_taken broadcast never fired on real hardware, forcing a fallback to requestPhoto directly from the button handler. Gemini Live's tool-call protocol sends an empty turn first when the model decides to call a tool, so a naive request/response wrapper resolved early and returned nothing — the request has to stay open across the tool round-trip. And a video frame sent before the question, instead of inside it, was invisible to the model, which then answered from imagination.

Accomplishments

A fleet that can honestly say UNKNOWN instead of guessing, and can reject its own agents' claims with a cited reason instead of rubber-stamping them. A property record that's provably rebuilt from facts on every request — no schema drift is even possible, because there's no separate "briefing" table to drift from. And a capture path that works identically whether it's a phone in a technician's hand or a pair of glasses that never leaves their face.

What we learned

Concurrency belongs in the database, not in application code — a per-(subject, predicate) advisory lock plus a partial unique index enforcing "one current fact" holds even if the Python regresses.

gcloud run deploy can exit 0 on a failed rollout. Every deploy script now re-describes the resulting revision and checks it's actually serving before calling itself done.

Name the job in every prompt, or the model picks one for you: a housekeeping turn that described a job by address instead of ID made the foreman write a technician's name into the wrong job at the same address. The write-gate caught it as a contradiction — which is the gate doing exactly its job — but the actual fix was disciplining the prompt, not patching the gate.

Vendor-documented device events can simply be dead on real hardware, and the honest fix is a fallback path plus a note in the code, not silence.

What's next

Wire the closer's A2A exit into our standalone A2A Bridge (Hedera-anchored audit trail for cross-org agent calls) — the card is already discoverable, the bridge is a consumer away. Port the glasses bridge off MentraOS's Cloud SDK, which the vendor is sunsetting, onto the successor Miniapp/Bluetooth SDKs once they're out of beta — the ~300-line bridge and device-agnostic intake contract mean this is a bridge rewrite, not a fleet change. Bring native audio output to the Live guidance brain once a raw-PCM playback path exists outside the Cloud SDK. And extend the closer's A2A exit toward a real downstream consumer — a home-warranty authorization flow or a second, independent agent fleet reading Foreman+'s memory instead of a person.

Bonus content

LinkedIn article on the build: https://www.linkedin.com/pulse/building-foreman-agent-fleet-where-nothing-becomes-truth-oskola-jxxme

Built With

• a2a
• bun
• cloud-run
• cloud-sql
• fastapi
• firestore
• gemini
• gemma
• google-adk
• lyria
• opentelemetry
• pgvector
• python
• typescript
• veo
• vertex-ai