SaiteAI

A project to make building managers gain time and money by providing agents assistance based on many takes of the actual construction site.

Comment

SiteManager GPT: Your AI Copilot for Construction Site Communication

Inspiration

Construction site management is a whirlwind of communication. Project managers and site managers are often buried under a constant stream of emails detailing progress updates, urgent issues, task requests, document exchanges, and more. Critical information can get lost, deadlines missed, and coordination becomes a major challenge.

We were inspired by the potential of Large Language Models (LLMs) and automation platforms to alleviate this burden. We envisioned an AI Copilot, SiteManager GPT, that could intelligently process incoming site communications, identify actionable tasks, and help managers stay organized and proactive, freeing them up to focus on critical on-site issues. Our goal was to build a functional prototype during this hackathon to demonstrate this concept.

What it Does

SiteManager GPT connects to a dedicated Gmail inbox for a specific construction project. Using a multi-step process orchestrated by N8N, it:

  1. Fetches New Emails: Automatically triggers when new emails arrive with a specific label (e.g., "Automat TP").
  2. Cleans & Prepares Data: Extracts key information (sender, subject) and decodes/sanitizes the email body.
  3. AI-Powered Analysis (Mistral AI): Calls the Mistral AI API (using mistral-large-latest) via an HTTP Request node to analyze the email content. The AI extracts structured information like project name, involved parties, potential deadlines, urgency, category, and key requested actions, outputting this as JSON.
  4. Data Parsing & Formatting: A custom N8N Code node parses the AI's JSON output and formats it for easier use and logging (e.g., converting date formats).
  5. Logging (Optional): Records the processed email information into a Google Sheet for traceability.
  6. Task Identification (Mistral AI): A second AI agent analyzes the extracted information (specifically the requested actions and context) to identify distinct, actionable tasks that need to be scheduled. It estimates details like task summary, description, and potential start/end times.
  7. Data Preparation for Calendar: The list of identified tasks is processed individually. A crucial Code node uses date libraries (like Moment.js, necessary due to N8N version constraints discovered during the hackathon) to reliably parse estimated dates/times and convert them into the strict ISO 8601 format required by Google Calendar, including fallback logic to ensure a valid date is always generated.
  8. Calendar Parameter Preparation (Mistral AI): A final, simpler AI agent takes the fully formatted task details (with validated ISO dates) and structures them into the exact JSON parameter object ({ Title, StartTime, EndTime, Description }) needed by the N8N Google Calendar node.
  9. Event Creation: A standard N8N Google Calendar node receives the parameters prepared by the agent (via an intermediate Set node for parsing) and creates the event in the user's primary Google Calendar.

This creates a pipeline where important actions identified in emails are automatically added as events to the site manager's calendar.

How We Built It

  • Orchestration: We chose N8N (self-hosted on AWS EC2 using Docker) as our central workflow automation platform due to its visual interface, flexibility, and robust node library.
  • AI Model: We leveraged the Mistral AI API (mistral-large-latest) for its powerful language understanding capabilities, accessed via N8N's HTTP Request node and Langchain Tools Agent nodes. We specifically configured it for JSON output mode for reliable data extraction.
  • Core Tools:
    • Google Workspace: We integrated directly with Gmail (trigger), Google Sheets (logging), and Google Calendar (event creation).
    • N8N Nodes: We extensively used Gmail Trigger, Set, Code (for data transformation, sanitization, and crucially, robust date formatting using Moment.js due to version limitations), HTTP Request, Tools Agent (Langchain integration), Split In Batches, IF, and Google Calendar nodes.
  • Architecture: We adopted a multi-agent pipeline approach within N8N. Each agent (powered by Mistral) has a specific role: initial information extraction, task identification, and final parameter preparation. Code nodes were essential for deterministic data transformation (especially date parsing/formatting) between agent steps. The final action (Calendar creation) uses a standard N8N node driven by data prepared by the last agent, ensuring reliability.
  • Hosting: The N8N instance runs within a Docker container on an AWS EC2 instance (Amazon Linux 2023), allowing us to utilize AWS resources.

Challenges We Ran Into

  1. EC2 & Docker Setup: Initial hurdles involved identifying the correct EC2 OS (AL2023) and finding the right commands to reliably install Docker Engine and Docker Compose V2 after standard methods failed. Debugging network connectivity (AWS Security Groups, URL syntax) to access the hosted N8N instance also took time.
  2. N8N Version & Date Libraries: Our specific N8N version (1.85.4) presented a major challenge as the expected built-in date libraries ($luxon, $moment) were not available globally in the Code node as anticipated. This required significant effort to create a robust date parsing and ISO 8601 formatting solution using the available $moment functions, including complex fallback logic to handle various date inputs or missing information. (Self-Correction: Based on final troubleshooting, Moment.js was used).
  3. AI Agent & Tool Interaction: Getting the N8N Tools Agent node to reliably pass all required parameters (especially Title and Description) to the connected Google Calendar node proved extremely difficult. Despite explicit prompts and tool descriptions, these parameters were consistently dropped. This forced a key architectural change: the final agent now prepares the parameters as a JSON object, and a standard Google Calendar node (not connected as a tool) executes the creation using reliable N8N expressions.
  4. AI Reliability & Prompt Engineering: Ensuring Mistral consistently returned valid JSON in the exact structure needed required careful prompt design, explicit instructions (e.g., "Respond ONLY with JSON"), and iterative refinement. Parsing the AI's output sometimes required cleaning up markdown artifacts.
  5. Google OAuth Setup: Navigating the Google Cloud Console to set up API keys, OAuth Consent Screens (handling "Testing" vs "Production" modes, adding Test Users), and correctly configuring Authorized Redirect URIs (using the EC2 public DNS name instead of the IP address) was a detailed process.

Accomplishments We're Proud Of

  • Successfully building a functional end-to-end pipeline integrating Gmail, multiple AI agents (Mistral), data transformation logic, and Google Calendar within N8N.
  • Overcoming the significant technical challenges related to the specific N8N version's date handling capabilities by implementing a robust $moment-based formatting solution.
  • Diagnosing and working around the limitations of the Agent-Tool interaction for the Google Calendar node.
  • Creating a system that demonstrably extracts tasks from email content and schedules them automatically.
  • Successfully deploying and running N8N on AWS EC2.

What We Learned

  • N8N is a powerful visual tool for complex workflow automation and AI integration, but specific node behaviors and available variables can be version-dependent.
  • Debugging multi-step agentic workflows requires careful inspection of intermediate data structures and execution traces. Agent text output alone isn't proof of action.
  • Reliable data transformation (especially date/time formatting and timezone handling) is often better handled by deterministic code (N8N Code node) than by relying solely on LLM prompts.
  • Agent-Tool interaction within frameworks like Langchain (as used by N8N nodes) can have limitations or require very specific prompting/configuration. Direct API calls or standard node executions can be more reliable for critical actions.
  • Cloud infrastructure setup (EC2, Docker, Security Groups) requires careful attention to detail.
  • Prompt engineering is key for getting consistent, structured output from LLMs.

Next Steps

  • Implement the human validation loop for actions beyond simple Calendar creation (e.g., drafting emails, updating external systems).
  • Handle email attachments (PDFs, images) using OCR or document parsing tools.
  • Improve task identification (handle multiple tasks more robustly, better date/time extraction).
  • Add more proactive features (e.g., reminders for incomplete tasks, daily briefings).
  • Develop a simple frontend/dashboard for better visualization and interaction (beyond Google Sheets/Calendar).
  • Refine error handling throughout the workflow.
  • Update to the latest N8N version to leverage $luxon and other improvements.

Built With

  • N8N: Workflow Automation (Self-hosted v1.85.4 on EC2)
  • Docker / Docker Compose: Containerization
  • AWS: EC2 Hosting
  • Mistral AI: LLM for analysis, task identification, parameter preparation (via API)
  • Google Workspace APIs: Gmail API (Trigger), Google Calendar API (Event Creation), Google Sheets API (Logging)
  • JavaScript (Node.js): Within N8N Code nodes for data transformation and date handling ($moment).

(Remember to add links to your GitHub repo and potentially a demo video on Devpost!)

Built With

  • n8n
Share this project:

Updates