Inbox Analytics Agent Architecture

Gemini Inbox Analytics Agent

A specialized, event-driven multi-agent suite designed to transform the Gmail experience from a passive list of messages into an intelligent, analytical "Command Center."

Inspiration

The modern inbox is a graveyard of "maybe later" emails. Users are constantly overwhelmed by automated notifications, system triggers, and newsletters that once held value but now only contribute to cognitive load. We were inspired to build a system that doesn't just sort mail, but actually understands the lifecycle of communication, helping users reclaim their focus by identifying patterns that traditional filters miss.

What it does

The agent analyzes email velocity, response metrics, and semantic content to provide deep insights and actionable cleanup strategies directly in the Gmail sidebar:

Response Time Analytics: Visualizes how quickly you respond to key contacts versus how long they keep you waiting, identifying "bottleneck" relationships.
Tiered Recommendations: Generates a personalized "Optimization Report" that categorizes clutter into three actionable tiers: Unsubscribe (recurring low-value mail), Archive (stale notifications), and Trash (redundant system triggers).
Semantic Search Tuning: Features a user-configurable "Similarity Threshold" slider, allowing users to fine-tune how aggressive the AI should be when identifying candidates for cleanup.

How we built it

We built a sophisticated, event-driven pipeline on Google Cloud:

Data Ingestion: Used Fivetran to synchronize operational data from MongoDB Atlas into BigQuery. This data is continuously updated on arrival by our Smart Email Manager Agent, creating a real-time bridge between operational sorting and analytical reporting.
Intelligence: Leveraged Vertex AI (text-embedding-004) to generate 768-dimension vectors for high-precision semantic matching across the entire stack.
Hybrid Search: Developed a custom SQL engine in BigQuery that combines vector similarity math with Keyword Boosting (+0.20 confidence bonus) to ensure accurate matching for short, technical emails (like "trigger" alerts).
User Interface: A native Google Workspace Add-on built with TypeScript and Webpack, providing a seamless dashboard experience inside Gmail.
Orchestration: Serverless Python (FastAPI) agents running on Google Cloud Run, communicating via the Model Context Protocol (MCP).

Challenges we ran into

The biggest hurdle was the "Dimensional Mismatch." During development, our agents were speaking different "languages"—some using 1024-dimension vectors and others using 768. This led to empty reports and zero search results. We solved this by performing a "Full Standard Realignment," migrating the entire stack and backfilling the BigQuery data to a unified Vertex AI standard. We also overcame "Transactional Drift" where Gmail's slow message indexing would trick our cleanup logic into deleting new labels.

Accomplishments that we're proud of

We are proud to showcase a robust Hybrid Search Engine. Pure AI vectors often struggle with very short strings, so we successfully implemented a "Keyword Match Bonus" system. By combining deep semantic understanding with exact keyword triggers, the agent provides recommendations that feel truly personalized and reliable, even for 1-word emails. Furthermore, the agent features a unique Cross-Agent Learning Loop: it improves its recommendations over time by incorporating User Preference Notes generated by our Self Arize-ing Manager Agent, which autonomously analyzes failed trace logs to refine the system's understanding of user intent.

What we learned

We learned that in a multi-agent system, Consistency is King. Aligning every part of the stack—from the database to the embedding model to the UI settings—on a single standard was the turning point for the project. We also gained deep experience in managing distributed state and "Self-Healing" data pipelines when syncing between operational and analytical stores.

What's next for Inbox Analytics Agent

Our roadmap includes:

Predictive Archiving (Auto-Pilot): Moving beyond "Recommendations" to an autonomous mode where the agent archives notifications based on historically confirmed user preferences.
Label-Level Intelligence: Implementing "Clutter Compression" and automated "Group Summarization" for semantic labels generated by the Smart Email Manager, providing high-level summaries of entire categories.
Architectural Specialization (A2A): Evolving our communication pattern so that all Gmail-native actions (like sending reports or archiving) are delegated to the Smart Email Manager Agent via Agent-to-Agent (A2A) calls, ensuring strict functional boundaries.
Deep BigQuery RAG: Enabling users to run complex semantic queries against their entire Gmail history using BigQuery as a long-term Retrieval-Augmented Generation (RAG) store.
Interactive HITL Reports: Adding "One-Click" action buttons (Unsubscribe/Archive/Delete) directly within the recommendation emails to provide an instant Human-in-the-Loop experience.
Contextual Focus Blocks: Integrating with Google Calendar to suggest "Focus Blocks" based on the times of day the user typically receives the highest volume of high-priority mail.