Middle

An autonomous B2B inbox agent turning Gmail orders into grounded, reviewable operations.

Comment

Inspiration

B2B back-office teams repeatedly read quote and purchase-order emails, identify the customer, copy SKU and quantity lines, check commercial thresholds, and prepare replies or ERP entries. The work is essential, but the information is fragmented between the inbox, attachments, product catalogs, customer records, and the ERP.

Middle turns that inbox traffic into a controlled operational workflow without removing the employee from consequential decisions.

What it does

Middle watches Gmail and processes each new message in two stages.

  1. A Google ADK classifier powered by Gemini 3.7 Flash assigns a controlled business tag and writes a visible Gmail label.
  2. Only a confident quote request or purchase order proceeds to multimodal extraction of the email body and supported PDF, PNG, or JPEG attachments.

The extracted customer is matched exactly by sender email and every product line by SKU against data imported by the user. Deterministic TypeScript then calculates totals, identifies missing data, and produces catalog-grounded free-shipping or cross-sell suggestions.

A native Gmail side panel displays the tag, extracted request, warnings, attachments, and recommendations. The operator can correct incomplete data, download an ERP-ready CSV, or explicitly create a Gmail reply draft. Middle has no send-email operation.

How it works

Gmail history events enter Google Cloud through an authenticated Pub/Sub push. A lightweight Cloud Run webhook verifies the Google identity, deduplicates the event, creates a deterministic Cloud Task, and acknowledges immediately.

The task executes the two-stage ADK/Gemini workflow, persists results and traces in Firestore, and performs only controlled Gmail API actions. Cloud Scheduler renews Gmail watches, Cloud KMS encrypts refresh tokens, and Secret Manager protects application secrets.

The model uses strict structured outputs validated with Zod and receives no mailbox-mutation tools. Email content and attachments are explicitly treated as untrusted data. Gmail labels, exact matches, monetary calculations, recommendations, drafts, and exports are owned by deterministic application code.

The agentic workflow

  1. Observe a Gmail history event.
  2. Classify the message with Google ADK and Gemini.
  3. Apply the corresponding visible Gmail label.
  4. Decide whether the message is a confident commercial request.
  5. Retrieve supported attachments only when needed.
  6. Extract the reference, customer evidence, SKUs, and quantities.
  7. Resolve the sender and products against imported reference data.
  8. Calculate totals and grounded recommendations.
  9. Persist a traceable result and expose controlled actions in Gmail.

This is an event-driven action workflow rather than a chat loop: Middle observes, routes, reasons, persists state, and prepares an operational result while keeping the employee in control.

Key features

  • Asynchronous Gmail processing with Pub/Sub, Cloud Tasks, and Cloud Run
  • Two-stage classification and conditional multimodal extraction
  • Visible color-coded Gmail labels
  • Exact company matching by sender email and product matching by SKU
  • Imported product and company reference data
  • Deterministic free-shipping optimization and complement-tag cross-sell
  • Native Gmail review and correction interface
  • Idempotent reply-draft creation
  • Signed ERP-ready CSV export
  • Seven-day operational analytics
  • Public deterministic fallback demo
  • No automatic email sending

Built with

  • TypeScript, Node.js 22, and Express
  • React and Vite
  • Google ADK for TypeScript
  • Gemini 3.7 Flash on Vertex AI with medium thinking
  • Cloud Run, Firestore, Pub/Sub, Cloud Tasks, and Cloud Scheduler
  • Cloud KMS and Secret Manager
  • Firebase Authentication and Google OAuth
  • Gmail API and Google Workspace Add-ons
  • Zod, Vitest, Testing Library, Supertest, and Playwright

Challenges

The first major challenge was authentication across three different callers: the React dashboard, Gmail/Workspace Add-on callbacks, and Google-to-Google infrastructure requests. Middle validates Firebase ID tokens for dashboard requests, Workspace Add-on identities for card actions, and Google OIDC tokens for Pub/Sub, Cloud Tasks, and Scheduler.

The second challenge was reliability. Gmail history notifications can be duplicated and asynchronous tasks can be retried, so Middle uses deterministic task names, per-message run keys, event receipts, and idempotent draft creation.

The third challenge was designing within the native Gmail Cards UI. Unlike a conventional embedded web application, the add-on is rendered by Google from server-produced Cards JSON, so the richer React interface remains in the dashboard while the Gmail panel focuses on fast contextual review and action.

What I learned

  • Separating classification from attachment extraction reduces unnecessary access and model work.
  • Agentic systems are safer when models produce typed decisions while deterministic code owns identity matching, prices, recommendations, and external mutations.
  • Google Cloud identity boundaries must be designed explicitly; one browser token cannot safely represent dashboard users, infrastructure services, and Workspace callbacks.
  • Native Gmail add-ons favor focused workflows over unrestricted visual customization.
  • Reliability and idempotence matter as much as model quality in asynchronous agents.

Accomplishments

  • A new Google-native TypeScript implementation built during the hackathon period
  • A live Gmail → Pub/Sub → Cloud Tasks → ADK/Gemini → Firestore pipeline
  • Conditional PDF/image extraction with strict schemas
  • Exact email/SKU grounding and deterministic recommendations
  • A native Gmail add-on with corrections, draft creation, and CSV export
  • Encrypted OAuth refresh tokens and authenticated service-to-service calls
  • A deployed dashboard, public fallback demo, architecture diagram, and reproducible README
  • Automated domain, security, API, workflow, add-on, and browser tests

Live verification

A synthetic purchase-order email and PDF completed the deployed workflow successfully. Gmail notified Pub/Sub, Cloud Tasks executed the ADK workflow, Gemini classified and extracted the request, Middle matched the company and SKUs, applied the visible purchase-order label, generated a catalog-grounded recommendation, created one reusable draft after explicit user action, and produced the ERP CSV. The message was never sent.

Try it

The public demo runs deterministic fixtures without requiring a Google account:

https://middle-dashboard-816686717960.europe-west1.run.app/demo

It reuses the production matching, recommendation, and Gmail-card rendering code, but does not call Gemini or mutate a mailbox. The complete authenticated Gmail workflow and deployment instructions are documented in the private judging repository.

Known limitations

  • Gmail is the only supported mail provider in this build.
  • Supported analyzed attachments are PDF, PNG, and JPEG.
  • ERP integration is represented by a downloadable CSV rather than a direct ERP write.
  • The OAuth consent screen remains in testing, so live judges need an authorized test account.
  • The public fallback demo is explicitly deterministic; Gemini and Gmail mutations are demonstrated through the authenticated live workflow.

Built With

  • cloud-firestore
  • cloud-kms
  • cloud-scheduler
  • cloud-tasks
  • express.js
  • firebase-authentication
  • gemini
  • gmail-api
  • google-adk
  • google-cloud-run
  • google-workspace-add-ons
  • node.js
  • playwright
  • pubsub
  • react
  • secret-manager
  • typescript
  • vertex-ai
  • vite
  • vitest
  • zod
Share this project:

Updates

Submission history