Inspiration

Email is where work arrives, but rarely where it gets completed. A single message can contain a deadline, a meeting request, and a question that needs a reply. Turning that message into a task, a calendar event, and a response requires repetitive manual work—and important details can easily be missed.

We built InboxPilot to close that gap. Our goal was not another inbox summarizer, but an agent that completes useful next steps while preserving a clear boundary around high-impact actions. Tasks and calendar events can be created automatically, but an AI-written email should never be sent without human approval.

What it does

InboxPilot is an event-driven AI inbox operator that monitors a connected Gmail account and converts incoming messages into organized actions.

For each email, the Gemini-powered agent can:

  • Extract one or more action items, including priorities and due dates.
  • Create meetings in Google Calendar when a concrete time is proposed.
  • Prepare context-aware replies as real Gmail drafts.
  • Perform several actions from the same message in one agent run.
  • Ignore newsletters and messages that do not require action.

The dashboard provides a transparent record of processed messages, selected tools, tasks, scheduled events, and drafts awaiting approval. Users can update task statuses, open Calendar events, review or discard drafts, and explicitly confirm before sending any email.

Each connected account receives an isolated workspace, and users can sign out or revoke InboxPilot’s Google access at any time.

How we built it

InboxPilot is built as two Python services deployed on Google Cloud Run:

  1. A public FastAPI web application handles Google OAuth, sessions, the dashboard, and human approval actions.
  2. A private agent service handles Gmail events, Gemini reasoning, and tool execution.

After OAuth authorization, InboxPilot registers a Gmail watch. Gmail publishes change notifications to Cloud Pub/Sub, which invokes the private Cloud Run service. Because Gmail notifications contain a history cursor rather than the full email, the service queries the Gmail History API to retrieve the changed messages.

Each message is processed by a Gemini 3.6 Flash agent using Google Agent Development Kit. The agent has direct access to tools for:

  • Creating tasks in Firestore
  • Creating Google Calendar events
  • Creating Gmail reply drafts
  • Updating task status

We chose a single tool-using agent because one email may require several actions. A flat agent can understand the message once and call every relevant tool without passing partial context through a chain of sub-agents.

Firestore stores user-scoped application state, Gmail history cursors, tasks, events, drafts, and processed-message records. Firestore transactions atomically claim message IDs before model invocation, making repeated Pub/Sub deliveries safe.

The production deployment also uses Google Cloud Scheduler for Gmail watch renewal and Google Secret Manager for sensitive configuration. OAuth credentials are encrypted before storage, browser sessions are signed, form actions include CSRF protection, and service-to-service requests are authenticated.

Challenges we ran into

One major challenge was understanding Gmail’s push-notification model. A notification does not include the new message—it only signals that mailbox history changed. We needed to persist a history cursor, query the History API, retrieve the relevant messages, and update the cursor safely.

Duplicate delivery was another important challenge. Pub/Sub and Gmail events can be delivered more than once, so a naive implementation could create duplicate tasks, events, and drafts. We addressed this by atomically claiming each Gmail message ID in Firestore before invoking the agent.

Multi-user support affected nearly every layer of the system. OAuth tokens, Gmail clients, sync cursors, Firestore records, idempotency keys, and dashboard queries all needed to share the same user context without leaking data between accounts.

We also had to design the right boundary between autonomy and control. Automatically sending an incorrect email has a much larger impact than creating a reversible task. InboxPilot therefore creates replies as Gmail drafts and requires a separate confirmation before sending.

Finally, model availability and event-driven timing required careful handling. We added retry behavior for transient Gemini errors, health endpoints for both services, and a manual synchronization option for predictable testing.

Accomplishments that we're proud of

We are proud that InboxPilot is a complete event-driven application rather than a standalone prompt demonstration.

The deployed system:

  • Processes real Gmail messages automatically through Pub/Sub.
  • Uses Gemini and Google ADK to select and execute multiple tools.
  • Creates real Google Calendar events and Gmail drafts.
  • Keeps generated communication behind a human approval gate.
  • Supports multiple Google accounts with isolated data and credentials.
  • Prevents duplicate processing with transactional idempotency.
  • Runs as separate public and private Cloud Run services.
  • Exposes the agent’s decisions and actions through a polished dashboard.

Most importantly, the product remains easy to understand: one email goes in, useful next steps come out, and the user stays in control.

What we learned

We learned that reliable agents require much more than a strong prompt. Identity, permissions, retries, idempotency, observability, and clear approval boundaries are essential parts of the agent’s behavior.

We also learned that the best amount of autonomy depends on the action. Tasks and events are visible and reversible, while outgoing communication deserves explicit review.

Finally, we found that a single agent with several well-defined tools was more predictable for this workflow than a complex handoff system. It reduced repeated reasoning and allowed the agent to complete every applicable action from one understanding of the email.

What's next for InboxPilot

Next, we want to make InboxPilot more personalized and collaborative by adding:

  • User preferences for tone, working hours, priorities, and default meeting length.
  • Smarter scheduling that detects conflicts and proposes alternative times.
  • Editable drafts and tasks directly inside the review workflow.
  • Daily and weekly summaries of outstanding commitments.
  • Team workspaces and assignment routing.
  • Additional action tools for documents, project trackers, and messaging platforms.
  • Stronger evaluation and observability for measuring extraction accuracy and tool-selection quality.

Our long-term vision is for InboxPilot to become a trusted operations layer for the inbox—handling routine coordination automatically while making every consequential action visible and controllable.

Built With

Share this project:

Updates

Submission history