Email Inbox Agent

I built an AI agent that autonomously manages email, analyzing messages, auto-scheduling meetings to Google Calendar, and creating Notion tasks. It saves hours by automating inbox workflows.

Comment

Inspiration

As an engineering student juggling classes, projects, and internships, I found myself drowning in hundreds of emails daily. I'd miss important deadlines buried in promotional spam, forget to schedule meetings mentioned casually in threads, and waste hours manually copying action items to my task manager. I realized this wasn't just my problem, professionals everywhere lose 2-3 hours daily to email management. What if AI could read my emails like a personal assistant and take action automatically? That question sparked the idea for an agentic email assistant that doesn't just analyze it executes.

What it does

Email Inbox Agent is an AI-powered assistant that transforms email from a time sink into an automated productivity system. It analyzes incoming emails using natural language processing, categorizes them by urgency (high/medium/low priority), and autonomously executes actions based on context:

  • Smart Email Analysis: Automatically categorizes and summarizes emails, highlighting urgent items and extracting key information
  • Calendar Integration: Detects meeting invitations in emails and creates Google Calendar events with proper times, attendees, and reminders
  • Notion Task Management: Extracts deadlines and action items, creating structured tasks in Notion with priorities and due dates
  • Natural Language Control: Users can ask "Schedule all meetings from this week" or "Create tasks for urgent emails" and the agent executes multi-step workflows
  • Smart Reply Generation: Generates contextual email responses in multiple tones (professional, casual, action-oriented)

The agent operates continuously, processing new emails in real-time and taking appropriate actions without manual intervention.

How we built it

Architecture:

  • Backend: Node.js with Express for API routing and server logic
  • AI Engine: Google's Gemini with function calling for natural language understanding and autonomous decision-making
  • Integrations: Gmail API for email access, Google Calendar API for scheduling, Notion API for task management
  • Frontend: Vanilla JavaScript with responsive CSS for the user interface
  • Authentication: OAuth 2.0 for secure Google account integration

Development Process:

  1. Email Pipeline: Built Gmail API integration to fetch and parse emails, extracting headers, body content, and metadata
  2. AI Analysis Layer: Implemented Gemini with custom function calling tools that allow the AI to autonomously decide when to create calendar events or Notion tasks
  3. Action Execution: Created tool functions for calendar scheduling, task creation, and Slack notifications that the AI triggers based on email content
  4. Caching System: Implemented JSON-based caching to avoid re-analyzing emails and improve performance
  5. Natural Language Interface: Built an agentic query system where users can issue commands in plain English and the AI translates them into multi-step actions

Key Technical Implementation:

const tools = [
  {
    name: "add_to_calendar",
    description: "Add event to Google Calendar when email mentions meeting",
    parameters: { summary, start_time, end_time, attendees }
  },
  {
    name: "create_notion_task",
    description: "Create task in Notion for action items in emails",
    parameters: { title, priority, due_date }
  }
];

The AI uses function calling to autonomously decide which tools to invoke based on email context.

Challenges we ran into

1. Function Calling Reliability: The AI sometimes generated malformed JSON or called wrong functions. Solved by implementing strict JSON validation, fallback parsing, and detailed tool descriptions.

2. OAuth Token Management: Handling expired tokens and refresh flows was tricky. Implemented automatic token refresh and graceful error handling with user-friendly re-authentication prompts.

3. Email Content Parsing: HTML emails contained excessive markup and nested structures. Built a robust HTML-to-text parser with regex cleaning and special character handling.

4. Rate Limiting: Gmail and Calendar APIs have strict rate limits. Implemented batching (processing 3 emails at a time), exponential backoff, and intelligent caching to minimize API calls.

5. Date/Time Extraction: AI struggled with relative dates like "tomorrow" or "next Friday". Enhanced prompts with explicit date formatting requirements (ISO 8601) and timezone handling.

6. Notion API Integration: Database property validation was strict, any mismatch caused failures. Debugged by adding comprehensive logging and property verification before API calls.

Accomplishments that we're proud of

True Agentic Behavior: Unlike simple chatbots, our agent makes autonomous decisions and takes real-world actions without human confirmation

Multi-Tool Coordination: Successfully implemented function calling where AI decides whether to create calendar events, Notion tasks, or both based on a single email

Natural Language Control: Users can issue complex commands like "Find all deadline emails and add them to calendar and Notion" and the agent executes the full workflow

Production-Ready Integration: Full OAuth implementation, error handling, and API rate limit management make this deployable for real users

Impressive Demo: The agent successfully scheduled 15+ calendar events and created 20+ Notion tasks during testing, proving practical value

What we learned

Technical Skills:

  • Agentic AI Architecture: Learned how to design AI systems that move beyond chat interfaces to autonomous action-taking agents
  • Function Calling: Mastered Gemini's function calling API for tool use and multi-step reasoning
  • API Integration Patterns: Deep understanding of OAuth flows, rate limiting, webhooks, and error handling across multiple services
  • Asynchronous Processing: Implemented proper async/await patterns, batching, and delay mechanisms for production-grade applications

AI Engineering:

  • Prompt Engineering: Crafted effective prompts that guide AI to make correct decisions about when and how to use tools
  • Context Window Management: Learned to truncate and prioritize information when dealing with long email threads
  • Fallback Strategies: Built robust fallback parsing when AI outputs don't match expected formats

Product Development:

  • User-Centric Design: Focused on solving real pain points rather than building features for their own sake
  • Iterative Testing: Continuous testing with real emails revealed edge cases that synthetic data never would
  • Production Mindset: Considered error handling, logging, and debugging from day one rather than as an afterthought

What's next for Email Inbox Agent

Immediate Enhancements:

  • Email Search & Filter: Natural language email search ("Find all emails from John about the budget")
  • Automatic Follow-ups: AI reminds you to follow up on unanswered important emails after 3 days
  • Multi-Language Support: Detect and translate emails from any language automatically
  • Sentiment Analysis: Flag angry customer emails or urgent complaints for immediate attention

Advanced Features:

  • Learning System: AI learns your preferences over time (which senders are important, preferred meeting times, etc.)
  • Email Thread Summarization: Condense long email chains into bullet-point summaries
  • Voice Commands: "Hey Agent, summarize my urgent emails" via Web Speech API
  • Analytics Dashboard: Visual insights on email volume, response times, and productivity metrics

Platform Expansion:

  • Mobile App: Native iOS/Android app with push notifications
  • Slack/Teams Integration: Send notifications to team channels for important emails
  • Trello/Asana Support: Additional task management integrations
  • Chrome Extension: Quick access to agent features while browsing Gmail

Enterprise Features:

  • Team Collaboration: Shared inboxes with AI-powered routing and assignment
  • CRM Integration: Auto-create contacts in Salesforce/HubSpot from email interactions
  • Compliance & Security: Email encryption, audit logs, and enterprise SSO

The vision is to make email completely invisible where you focus only on strategic decisions while AI handles all coordination, scheduling, and routine communication automatically. Email Inbox Agent is just the beginning of truly agentic productivity assistants.

Built With

Share this project:

Updates