Enterprise AI Core

A zero-trust, multi-agent platform orchestrating specialized Gemini agents to automate enterprise engineering tasks, SoW creation, and cloud assessments with direct doc exports.

Comment

💡 Inspiration

Enterprise cloud architects and engineers spend a disproportionate amount of high-value engineering time on repetitive administrative toil: drafting detailed Statements of Work (SoWs), calculating granular effort estimations, analyzing multi-pillar cloud governance telemetry, and synthesizing technical diagrams.

At the same time, enterprise adoption of Generative AI is frequently blocked by security, data residency, and identity governance concerns. I built Span Enterprise AI Core to bridge this gap—creating an intelligent, enterprise-grade intermediary that securely connects corporate Active Directory identity, enterprise data, and state-of-the-art Google Gemini models into a unified, compliant multi-agent ecosystem.

🚀 What It Does

Span Enterprise AI Core is a decoupled, multi-agent platform designed to automate heavy-lifting engineering workflows while enforcing strict zero-trust security:

  • Smart Orchestration & Auto-Routing: A central FastAPI gateway evaluates user intent in real-time (using zero-temperature Gemini models) against the user's active Microsoft Entra ID role permissions, routing queries to the exact specialized agent needed.
  • Specialized Agent Microservices:
  • SoW Generator Agent: Ingests complex project requirements and outputs immutably structured, dual-language (English/Croatian) Statements of Work mapped to branded corporate templates.
  • Effort Estimator Agent: Analyzes SoW documents to automatically calculate precise engineering timelines, role allocations, and workload estimates exported directly to .xlsx.
  • CAST Assessment Agent: Computes deterministic SHA256 deltas across multi-run cloud assessments, calculating 6-pillar posture scores (Security, Cost, Reliability, Performance, Operational Excellence, Governance) with executive synthesis.
  • Code Assist & Architecture Agent: Powered by dual-engine Gemini reasoning for complex cloud infrastructure automation, Terraform, and database scripting.
  • Deep Research Agent: Performs autonomous multi-step technical investigations featuring Human-in-the-Loop interactive planning phases.

  • Diagram Documenter & SAP Research Agents: Specialized microservices for multimodal architecture diagram analysis and SAP knowledge base parsing.

  • Branded Templated Exports: Automatically strips internal agent reasoning artifacts (<thinking>, <planner>) and transforms live Markdown streams into fully stylized, corporate .docx and .xlsx deliverables.

  • Contextual Workflow Handoffs: Dynamically surfaces UI handoffs between agents (e.g., passing a generated SoW directly into the Effort Estimator) for seamless multi-step workflows.

🛠️ How We Built It

The platform is architected as an ephemeral, containerized microservices ecosystem deployed on Google Cloud Run (Gen 2):

  • Frontend Client (frontend-ui): Built with Next.js 14+ (React) and Tailwind CSS. Features a custom ReadableStream NDJSON reader for character-by-character neural output streaming, dynamic feature-flagging based on entitlement APIs, and native Mermaid chart diagramming.
  • Central Orchestrator & Agents (fastapi): Developed with Python (FastAPI) and asyncio. Utilizes Google Cloud Run Identity (OIDC) tokens procured via the Google Metadata Server for secure, keyless service-to-service communication.
  • AI Foundational Models: Deployed on Google Vertex AI (ensuring strict EU data residency compliance), dynamically pairing workloads to optimal models:
  • gemini-3.1-pro-preview for high-reasoning tasks (Deep Research, Executive Synthesis, Architecture Planning).

  • gemini-3.7-flash for high-throughput, low-latency tasks (Fast Chat, NDJSON streaming, Document Summarization).

  • Identity & Zero-Trust RBAC: Sits behind Google Cloud Identity-Aware Proxy (IAP), Cloud Armor (WAF/DDoS), and integrates with Microsoft Entra ID (Azure AD) via Microsoft Graph (/transitiveMemberOf) with pagination handling to enforce fail-closed authorization.

  • Data & Semantic Caching: Utilizes Google Cloud Firestore for session persistence and geometric vector embeddings (cosine-similarity semantic caching) to eliminate duplicate compute costs.

  • CI/CD & Observability: Automated with GitHub Actions via OIDC Workload Identity Federation and monitored end-to-end via OpenTelemetry and custom token-observability dashboards.

🧗 Challenges We Ran Into

  1. Deeply Nested Enterprise RBAC: Handling transitive group memberships for enterprise users belonging to hundreds of overlapping Active Directory groups required implementing paginated Microsoft Graph token interception without adding latency to the gateway handshake.
  2. Clean Artifact Generation from Streaming AI: Real-time NDJSON streaming provides instant UI feedback, but exporting production-ready client deliverables required building a real-time parsing engine that strips internal agent chain-of-thought blocks (<thinking>, <qa_critic>) while preserving formatting, tables, and Mermaid charts for .docx compilation.
  3. Deterministic Cloud Assessment Deltas: Processing multi-run CAST assessment ZIP archives required computing exact SHA256 FindingHash deltas between historical telemetry runs before feeding the multi-pillar diffs into Gemini for contextual risk synthesis.

🏆 Accomplishments That We're Proud Of

  • Drastic Overhead Reduction: Successfully eliminated hours of manual administrative effort per project for cloud architects and engineers through automated SoW generation, effort estimation, and cloud assessment reporting.
  • True Zero-Trust Compliance: Built a production-ready enterprise GenAI platform that guarantees EU data residency, enforces zero-knowledge perimeter defenses, and completely eliminates long-lived static API keys.
  • Sub-Second Time-To-First-Byte (TTFB): Combined Gen-2 Cloud Run HTTP/2 streaming with Vertex AI high-throughput models and semantic caching to achieve near-instantaneous streaming responses.

📚 What We Learned

  • Hybrid Model Tiering is Essential: Using a single model for all tasks is inefficient. Routing deep-reasoning synthesis to gemini-3.1-pro and streaming tasks to gemini-3.7-flash creates the optimal balance between response quality, latency, and cost.
  • Agentic Specialization Beats Monoliths: Domain-specific agents with scoped system prompts, dedicated tool injections, and intelligent workflow handoffs deliver vastly superior accuracy and user trust compared to generic, monolithic chatbots.

🔮 What's Next for Span Enterprise AI Core

  • Tender Review Agent: A dedicated pre-sales intelligence microservice to ingest complex RFP/tender packages, evaluate organizational fit and technical compliance, and generate concise bid/no-bid qualification reports with exact source citations.
  • Branded PowerPoint Presentation Agent: An automated generation pipeline that transforms topics and engineering outlines into fully stylized .pptx decks adhering strictly to Span's corporate visual identity, color palettes, typography, and custom icon sets.
  • xWiki & CCoE Knowledge Base Integration: Deep bi-directional synchronization with Span’s internal xWiki repository (housing client profiles, project histories, architecture blueprints, and Cloud Center of Excellence guidelines) to ground SoW drafting, research, and deck creation, as well as autonomously publish new architecture chapters.
  • SharePoint Online & Internal Tool Ecosystem: Native connectors for SharePoint Online and Span’s internal operational tooling to enable seamless bi-directional data flow, telemetry retrieval, and artifact publishing.
  • Inline @-File Context Referencing: Interactive chat tagging allowing engineers to directly target and ground agent reasoning in specific workspace files, telemetry packages, and architectural documents on the fly.

💻 Built With

  • google-cloud-run
  • google-vertex-ai
  • gemini-3.1-pro
  • gemini-3.7-flash
  • fastapi
  • python
  • next.js
  • react
  • tailwind-css
  • google-cloud-firestore
  • google-cloud-armor
  • identity-aware-proxy
  • microsoft-entra-id
  • opentelemetry
  • docker

Built With

Share this project:

Updates

Submission history