Boutique-Co-Pilot

A proactive, multimodal AI shopping assistant for e-commerce, built on GKE Autopilot and Vertex AI Gemini.

Comment

What inspired you?

My inspiration for the "Boutique Co-Pilot" came directly from the theme of the GKE Turns 10 Hackathon. Ten years of GKE represents a decade of simplifying complex application delivery. I wanted to celebrate this by tackling a common, real-world challenge: how do you add next-generation AI capabilities to an existing, complex microservices application without a painful rewrite?

Traditional e-commerce is often a static and lonely experience. My vision was to transform the Online Boutique demo into a dynamic, interactive space where an AI assistant actively accompanies the user, understanding their journey just like a helpful associate in a real store. I was inspired by the power of modern multimodal models like Gemini and wanted to see if I could create a truly context-aware "co-pilot" using the scalable, managed foundation of GKE Autopilot.

How you built your project

I followed a methodical, iterative approach, building the project in distinct phases on a GKE Autopilot cluster.

1. Foundation & Baseline: First, I established the baseline by deploying the original Online Boutique application to a clean GKE Autopilot cluster. This provided the "world" in which my Co-Pilot would live.

2. Building the "Brain" - The Co-Pilot Agent: Next, I created the core of the project: an independent microservice in Python using the FastAPI framework. This "agent" was designed with endpoints to receive UI events (/event) and handle conversations (/chat). I containerized it with a Dockerfile and ensured it could communicate with the Vertex AI Gemini API.

3. The UI - A Non-Invasive Integration: A key goal was to avoid modifying the original application's source code. To achieve this, I wrote a vanilla JavaScript file (chat.js) that programmatically creates and injects a full chat UI into the DOM of any page it's loaded on. I then built a new, custom Docker image for the frontend service, based on the original, but with one key change: it included this new chat.js file and a modified footer.html to load it.

4. Layering on Intelligence: With the basic chat loop working, I incrementally added layers of context-awareness:

  • Page Scraping: The chat.js script was enhanced to read the DOM of the current page, extracting product names, prices, and descriptions.
  • Cart Awareness: It learned to read the contents of the shopping cart page, including items, quantities, shipping, and the total cost.
  • Conversational Memory: I implemented session-based memory using sessionStorage in the browser, allowing the conversation history to persist across page navigations.
  • Multimodal Vision: The final and most exciting feature. The frontend sends the product image URL to the backend. The agent then downloads the image and includes it in a multimodal prompt to Gemini, allowing the AI to answer questions based on what it "sees."

5. Networking & Permissions: To connect everything, I deployed a GKE Ingress to route traffic intelligently based on the URL path. For security, I implemented GKE Workload Identity to provide the agent pod with secure, passwordless credentials to access the Vertex AI API.

The challenges you faced & what you learned

This project was an incredible, real-world learning experience. The biggest challenges were not in the AI logic, but in the integration and cloud-native plumbing.

  • Challenge 1: The Legacy Frontend Conflict: The most persistent challenge was a series of subtle JavaScript bugs. The original application uses older versions of jQuery and Bootstrap. My modern, vanilla JavaScript code was conflicting with the page's existing scripts, causing issues with event listeners and script loading order. It took a deep, methodical debugging process—using the browser's developer tools to inspect the DOM, event listeners, and console—to finally resolve this by taking full control of the script loading order and using jQuery for event delegation to ensure compatibility.

  • Challenge 2: The Cloud-Native "Gotchas": Making everything work reliably on a fresh cluster was a huge challenge. I learned firsthand about the complexities of cloud networking and identity.

    • Permissions: Configuring Workload Identity correctly is a multi-step process. A single missing IAM binding or, as we discovered, a missing Project ID in a Kubernetes annotation, can cause authentication to fail silently.
    • Networking: The 502 Bad Gateway errors we encountered taught me a crucial lesson about GKE Ingress. The problem wasn't a crashing pod, but a missing firewall rule that prevented the Google Cloud Load Balancer's health checks from reaching the pods.

What I Learned: My key takeaway is that building a powerful AI feature is only half the battle. The true art of cloud-native engineering lies in the details: robust integration, methodical debugging, and a deep understanding of networking and IAM. I learned that prompt engineering is an iterative process of "calibrating" an AI's personality, moving from a simple instruction-follower to a truly intelligent and conversational partner. This hackathon has been an invaluable, end-to-end lesson in building and debugging a real-world, scalable AI application.

Built With

  • artifact-registry
  • browser-session-storage
  • cloud-build
  • css3
  • docker
  • fastapi
  • gemini-1.5-flash
  • git
  • github
  • gke-autopilot
  • gke-workload-identity
  • go
  • google-cloud-aiplatform
  • google-cloud-load-balancing
  • html5
  • in-memory-python-dictionary
  • javascript
  • jquery
  • kubectl
  • kubernetes-api
  • mermaid.js
  • pydantic
  • python
  • uvicorn
  • vanilla-javascript
  • vertex-ai
  • vertex-ai-api
  • yaml
Share this project:

Updates