Inspiration

In California, modifying your car's engine is illegal unless every part is approved by the California Air Resources Board. Those approvals — Executive Orders — live in more than six thousand PDFs on a government website: some typewritten fifty years ago, some published last week. There is no search engine. Years ago I built a regex/OCR pipeline to index them for my site SmogLegal, and it taught me a hard lesson: on its best days it got two-thirds of the part numbers wrong, and it structurally couldn't answer the only question that matters — "what's legal for MY car?" No regex survives five decades of document formats. An agent that actually reads them can.

What it does

CarbLegal is an autonomous agent workforce on Google Cloud that reads every CARB Executive Order and keeps reading. Every morning, Cloud Scheduler wakes an ADK 2 graph workflow: a Scout diffs the state's registry, a healer forgives yesterday's transient failures (and re-downloads corrupt PDFs), an Extractor reads each scanned document with Gemini 3.7 Flash's vision, an Auditor critiques every extraction — with the final pass-or-escalate call made by a deterministic gate, plain code, not AI — and a Matchmaker joins parts to 25,000 real vehicles. The result: 6,200+ documents read, 200,000+ part-to-vehicle answers that never existed anywhere, supersession chains detected by reading the documents' own "supersedes and cancels" clauses (600+ found — the old document never tells you it's been replaced), and a search that CARB never built: pick your car, see every legal part, one click to the state's own signed PDF.

How we built it

Gemini 3.7 Flash on Vertex AI does all reasoning — extraction (native-PDF rung with an image-render fallback and truncation detection), selective critique, and ambiguous-match resolution. The daily orchestration is a real google.adk.workflow graph; the GenAI SDK powers the reasoning call sites. Firestore holds a leased, checkpointed work queue — kill the process mid-job and the next run picks up exactly where it stopped. The historical backfill ran through Vertex batch inference at half price, through the same deterministic gate as the daily path. Cloud Run hosts the pipeline and an Astro dashboard (live agent console, run traces, EO browser with legacy-vs-agent diffs and supersession lineage, a review queue where human corrections resume the pipeline mid-stream, and the vehicle search). Everything is ADC — zero API keys. Claude Code was used as a build assistant throughout; the legacy regex system's output survives only as the baseline the agent is measured against.

Challenges we ran into

The honest list is long, and each entry made the system better. A queue live-lock taught us to trust transactional leases over clever sharding. Vertex 429 storms became the self-healing node with a three-strikes rule. Cloud Scheduler's silent 180-second deadline became a graceful in-process time cap. Our biggest fight was with our own deterministic gate: it kept escalating documents that were fine — motorcycles it didn't know, 1928 vintage-car coverage, 58-liter industrial Caterpillar engines, part numbers like "GO Kit" — because its rules were car-brained. We rebuilt the rules against reality four times, and a deterministic regrade swept 550+ false escalations out of the human queue, leaving 0.7% of documents that genuinely deserve human eyes. We also caught our own evaluation inflating a score — the association metric ignored rows the agent missed — and re-scored ourselves worse, on the record.

Accomplishments that we're proud of

98% part-number accuracy against a human-adjudicated answer key, versus 56% for the old regex — which scored zero on connecting parts to cars, because it never could. A judge can clone the repo and reproduce the whole evaluation with one pytest command, no credentials. The agent found 174 brand-new approvals California published while we were building — on its own. And a 1975 document that contradicts itself (page one says "1974 and later," the body says "and older") was caught by the Auditor, refused by the gate, and settled by a human with one click — the system knowing what it doesn't know.

What we learned

Deterministic beats clever: every place an LLM opinion was replaced by a rule — the gate, the failure classifier, the row-count tripwire — got more testable and more trustworthy. Rules must learn from reality, not intuition: our gate's false-escalation rate fell 50× once its bounds came from real documents instead of assumptions. A language model can't be its own alarm clock; scheduling is infrastructure. And the document's identity is a datapoint — a garbled header never outranks the file you fetched.

What's next for CARB Legal

A second-tier arbiter that re-reads escalated documents cold and auto-resolves only on independent agreement, shrinking the human queue further without removing the human. A completeness rung that re-extracts giant multi-page appendices in chunks when the row-count tripwire fires. Moving the runner onto Agent Engine with Cloud Trace export, curating run events into BigQuery, and driving adk optimize from our golden eval — the fitness function is already built. Then: feeding this index back into SmogLegal, so every Californian can finally answer "is this part legal for my car?" in two clicks.