OmniFlow AI: Smart Venue Orchestration

OmniFlow AI bridges physical operations and DevOps. It uses Gemini to analyze live crowd telemetry, autonomously reroute traffic, and dispatch real-time IT tickets via Model Context Protocol.

Comment

Inspiration

Modern massive-scale venues—like stadiums, mega-conferences, and carnivals—are chaotic. When a physical bottleneck occurs (like a broken turnstile or a blocked corridor), crowd density spikes dangerously. Usually, venue operators rely on passive cameras and radios, reacting after the crowd is already frustrated or unsafe. We wanted to ask: what if a venue was a self-healing system? What if an AI agent could analyze spatial telemetry in real-time, autonomously reroute crowds via digital signage, and instantly orchestrate IT repairs?

What it does

OmniFlow AI is an intelligent venue operating system. It ingests live spatial telemetry from various physical topologies. When it detects a critical anomaly, it dispatches the data to Gemini 3.1 Flash Lite for live spatial execution planning.

Gemini acts as an active orchestrator. It calculates a dynamic rerouting path for the physical crowd, generates a payload to display on physical digital screens, and queues a Human-in-the-Loop (HITL) authorization request for the operator.

But it goes deeper. Using the Model Context Protocol (MCP), OmniFlow bridges physical operations with enterprise DevOps:

  1. Historical Context: It queries MongoDB Atlas for historical incident matches to inform Gemini's decisions, then saves the new incident to continuously train the system.
  2. Enterprise IT Orchestration: It uses the MCP to autonomously generate a live GitLab engineering ticket, injecting Gemini's root-cause deductions straight into the hands of the software developers.
  3. Data Security: It pushes aggregated telemetry to a secure Elasticsearch cluster, utilizing a custom enterprise security gateway to protect API keys while proving data flow.

How we built it

  • Frontend & Visualization: We built the dashboard using Next.js, React, and TypeScript. We used React Flow to render interactive, node-based topological graphs of the venues, and Framer Motion for sleek, enterprise-grade micro-animations.
  • AI Brain: The core logic is powered by the new @google/genai SDK using the ultra-fast Gemini 3.1 Flash Lite model, configured with strict JSON schemas to guarantee deterministic routing decisions.
  • MCP Integrations: We implemented a robust backend API route that spawns local standard IO transport processes to securely talk to official MCP servers (GitLab, Elasticsearch, MongoDB).
  • Deployment: The entire architecture is containerized and deployed to Google Cloud Run for serverless, auto-scaling performance.

Challenges we ran into

Integrating multiple official MCP servers within a Next.js API route was complex. We had to carefully manage Node.js child processes and standard IO streams to ensure the MCP clients could securely communicate with the host servers without leaking environment variables. Furthermore, ensuring Gemini returned perfectly structured spatial arrays and signage payloads every single time required rigorous prompt engineering and strict JSON schema configurations.

Accomplishments that we're proud of

We successfully demonstrated "IT/OT Convergence." OmniFlow AI proves that an AI agent shouldn't just be a chatbot; it can be a central nervous system that simultaneously commands physical hardware (digital signage) and digital enterprise tools (DevOps issue tracking).

What we learned

We learned the sheer power of the Model Context Protocol (MCP). By giving Gemini direct tool access to standard enterprise platforms, the ceiling of what an AI agent can do is exponentially higher. We also learned how to balance full autonomy with necessary safety measures by implementing our Human-In-The-Loop authorization gate.

What's next for OmniFlow AI

We plan to integrate real-time computer vision feeds so OmniFlow can generate its own telemetry from physical cameras. We also want to expand the MCP suite to include automated Slack alerts to security personnel and dynamic API integrations with local city transit systems to predict incoming crowd surges before they even enter the venue!

Built With

  • elasticsearch
  • framer-motion
  • gemini-3.1-flash-lite
  • gitlab-api
  • google-cloud-run
  • model-context-protocol-(mcp)
  • mongodb-atlas
  • next.js
  • react
  • tailwind-css
  • typescript
Share this project:

Updates