Mall Operations Brain

An autonomous AI operations cockpit with multi-step reasoning agent . Single platform as: Manager Cockpit for operational analytics Shopper Kiosk for customer-facing .

Comment

Inspiration

While online e-commerce platforms track every click and utilise advanced analytics to personalise the shopping experience, physical malls often operate on delayed reports, fragmented data silos, and intuition.

I was inspired to build a single, intelligent "brain" for physical retail spaces. I wanted to create an agent that doesn't just answer questions, but actively does work—providing real-time shopper navigation, automating hyper-local tenant campaigns, and streamlining facility operations, just as the hackathon prompt suggested.

What it does

Mall Operations Brain is an autonomous AI operations cockpit for brick-and-mortar retail, built with Google Cloud Agent Builder and powered by the Elastic Model Context Protocol (MCP) server.

It serves two distinct personas from a single unified platform:

  1. The Manager Cockpit: A powerful dashboard that allows operators to run complex diagnostics (e.g., "Which stores underperformed this month?"), compose targeted marketing campaigns using semantic search on historical data, and triage facility maintenance issues.
  2. The Shopper Kiosk: A customer-facing portal acting as a personal co-pilot. It offers mathematically optimal backtrack-free itinerary planning, live deal discovery via an autonomously curated dashboard, and interactive coupon activations.

Rather than being a simple chatbot, it is a multi-step reasoning agent that connects, correlates, and acts on live enterprise data across 7 different Elasticsearch indices to accomplish tasks under human oversight.

How I built it

This project was built to showcase the immense power of combining Gemini's reasoning with Elastic's robust data infrastructure:

  • The Brain: Built on the Google ADK (Agent Development Kit) utilizing Gemini 3 for advanced reasoning, planning, and tool use.
  • The Superpowers (Elastic MCP): I integrated the Elastic MCP Server to give our agent its superpowers. Instead of hardcoding API endpoints, the agent dynamically discovers search, index listing, and analytical tools at runtime via the Model Context Protocol.
  • The Memory & Analytics Layer: Elasticsearch serves as both our database (housing 7 indices including tenant-sales, foot-traffic, and customer-coupons) and our "living memory layer". The agent natively uses ES|QL for complex analytical queries and a Hybrid Search engine combining lexical BM25 and dense vector KNN for semantic understanding.
  • Orchestration: Complex, state-changing operations (like issuing a coupon and logging the conversion) are encapsulated into Elastic Serverless Workflows, which the agent triggers deterministically.
  • UI & Deployment: A FastAPI backend streams reasoning steps via SSE to a Next.js frontend. The agent itself is seamlessly deployed to the Vertex AI Agent Engine using Google's agents-cli.

Challenges I ran into

  1. Complex Analytical Queries: LLMs often struggle to write perfect SQL or query complex analytical databases without hallucinating. I solved this by exposing Elasticsearch's ES|QL API via MCP. I wrapped the responses to return structured markdown tables, allowing Gemini 3 to effortlessly reason over aggregated sales data and foot-traffic metrics.
  2. Safe Execution of State-Changing Actions: Actions like activating a shopper coupon require querying one index (promotions), writing a token to another (coupons), and logging analytics to a third (foot-traffic). Letting an LLM run these steps individually is risky. I solved this by utilising Elastic Serverless Workflows. The agent simply triggers the workflow via a single tool call, and Elastic safely executes the deterministic chain.
  3. Context Window Overload: Tasking a single agent to analyze weather, foot traffic, and sales simultaneously to curate a live dashboard led to token bloat. I solved this by creating an Autonomous Pulse Dashboard that orchestrates 3 distinct AI subagents every 5 minutes, each utilising different Elastic tools, before passing findings to a "Synthesiser" subagent.

Accomplishments that we're proud of

  • Dynamic Tool Discovery: Successfully using the Elastic MCP Server to let the agent auto-discover its tools, proving that MCP is the future of agentic data access.
  • Living Memory: Turning Elasticsearch into a read-write memory layer where every agent action (like a coupon activation) is indexed immediately, creating a real-time feedback loop for future agent reasoning.
  • Zero-Hallucination Pathfinding: By offloading the complex spatial math of route planning from the LLM to a native Python tool querying a geo_point index in Elasticsearch, I achieved mathematically optimal itineraries.
  • Production-Ready Deployment: Using agents-cli to package and deploy our ADK agent directly to Google Cloud, abstracting away the complex infrastructure overhead.

What I learned

  • MCP is a Game Changer: Using the Model Context Protocol (MCP) to let the agent auto-discover its capabilities from the Elastic cluster is significantly more robust and scalable than manually wiring dozens of REST APIs.
  • ES|QL + LLMs = Magic: ES|QL's piped syntax is incredibly intuitive for LLMs to generate. When combined with Gemini 3's reasoning, the agent can compose its own ad-hoc analytical queries on the fly with shocking accuracy.
  • Elasticsearch Workflows are Deceptively Simple: I learned that Elastic Serverless Workflows are incredibly easy to define and use, yet they provide the powerful, deterministic execution engine required to handle complex, multi-step agent actions safely.
  • Delegation is Key: Complex AI systems function best when the main LLM orchestrates specialized tools (like Elastic Workflows) and subagents, rather than trying to perform deterministic database operations natively.

What's next for Mall Operations Brain

  • IoT Sensor Integration: Expanding the use of Elasticsearch to ingest live feeds from physical IoT occupancy and temperature sensors, giving the agent second-by-second environmental awareness.
  • Augmented Reality (AR) Wayfinding: Extending the shopper itinerary feature with an AR mobile app integration, pulling real-time geo_point data from Elastic to guide users through the physical mall.
  • Predictive Diagnostics: Leveraging Elastic's Machine Learning features directly alongside the agent to shift the Manager Cockpit from reactive analysis to proactive, predictive facility management.

Built With

Share this project:

Updates