Custody Loop

Inspiration

Waste traceability often breaks at the last handoff: evidence arrives from different actors, weights disagree, and an operator must reconstruct what is missing before a load can be released or reported. Custody Loop turns that completed-collection event into a durable, inspectable workflow that closes the exception instead of merely describing it.

What it does

Custody Loop handles three synthetic operational paths:

1. A clean four-stage custody chain releases automatically with one evidence-linked packet.
2. A missing receiver handover recovers from a transient extraction failure, blocks release, creates one evidence-specific task, ignores a duplicate trigger, accepts the correction, resumes the same workflow, and releases with the original exception marked resolved.
3. A 5.39% receiver-weight conflict stops at APPROVAL_REQUIRED. Release needs a named officer and a concrete reason, and the approved exception remains in the packet.

The console makes the autonomous work visible through the custody route, evidence manifest, open task, retry counter, current state, audit ledger, approval boundary, and packet digest.

How we built it

• Gemini 3.5 Flash on Vertex AI performs schema-constrained multimodal extraction from synthetic PDF and image evidence. Trusted event-envelope fields stay separate from model claims.
• Google Agent Development Kit (ADK) provides a real Agent and App with six narrow tools for starting, inspecting, correcting, approving, replaying, and resuming workflows.
• Cloud Run hosts the FastAPI service and operator console at the public .run.app URL with a dedicated runtime identity and scale-to-zero operation.
• Firestore transactionally persists typed workflow checkpoints, tasks, exceptions, audit events, retry history, idempotency keys, and packet hashes.
• Python, FastAPI, Pydantic, ReportLab, and vanilla JavaScript implement the API, typed domain model, deterministic rules, PDF proof packet, and responsive console.

The model-assisted perception boundary is deliberately decoupled from custody control. Gemini reads uncertain evidence; deterministic code remains the release authority.

Challenges we ran into

The hardest problem was making recovery truthful. A retry counter in the UI is not enough: the workflow must persist each attempt, avoid repeating completed tools, survive reconstruction, and suppress duplicate collection events. We solved that with stable workflow and action keys, monotonic store versions, bounded retry states, and same-ID resume behavior.

The second challenge was constraining model authority. Evidence extraction benefits from Gemini, but a generative model should not decide whether a custody chain is operationally releasable. We separated fuzzy extraction from deterministic rules and added a narrow human boundary only for documented weight variance.

The final challenge was proving the system in under four minutes. The published video uses one continuous live Cloud Run execution, shows all three cases, preserves waits and retries, and displays the public .run.app path plus the live health response. Narration was recorded separately for audio clarity; the live run was not cut or reordered.

Accomplishments that we are proud of

• One complete block-correct-resume-release loop instead of a broad prototype.
• Persisted retry, replay suppression, correction, approval, and packet release visible in one operator surface.
• Eight live Vertex-backed evidence extractions in the hero flow with provider, model, and field provenance.
• A Firestore record whose packet SHA-256 matched the downloaded PDF.
• A dedicated Cloud Run runtime identity with no Gemini API key mounted in the deployed revision.
• Eighteen deterministic tests covering rules, API behavior, persistence, idempotency, retry exhaustion, reset isolation, and the Google ADK tool surface.

What we learned

Agentic reliability comes from explicit state and bounded tools, not from a longer prompt. The most useful model boundary was perception: Gemini normalized messy evidence, while typed code governed decisions and side effects. We also learned that long-running is a workflow property, not a continuously running process; Firestore checkpoints let Cloud Run scale to zero and safely continue when the next event arrives.

What is next

The current vertical slice intentionally uses synthetic evidence and keeps released packet bytes in Cloud Run ephemeral storage while persisting the packet hash in Firestore. A production version would place packet bytes in Cloud Storage, replace the synthetic trigger with an authenticated collection event, add role-based approval, and run a larger adversarial extraction evaluation before connecting to a real reporting or ERP boundary.

Disclosure and data sources

• The application and synthetic fixtures were created during the hackathon submission period.
• All collection records, organizations, times, routes, weights, evidence images, and PDFs are synthetic and generated by this repository.
• Barlow Condensed SemiBold is redistributed under the SIL Open Font License; the license text is included under licenses/.
• Open-source Python packages and versions are declared in pyproject.toml and locked in uv.lock.
• AI coding and visual-composition assistance was used during development; product claims were verified against tests and the deployed service.
• The source code is offered under the MIT License.
• The demo is one continuous live Cloud Run execution with separately recorded English narration for audio clarity. No live steps were cut or reordered.

Built With

• cloud-run
• css
• fastapi
• firestore
• gemini-3.5-flash
• google-adk
• html
• javascript
• pydantic
• python
• reportlab
• sqlite
• vertex-ai