Inspiration
Customer service in e-commerce is fundamentally broken. It is overwhelmingly reactive—brands wait for customers to get angry about a delayed package before taking action. Furthermore, human agents spend countless hours toggling between Shopify, FedEx, and company policy documents just to issue a $10 store credit.
We were inspired to build JARVIS, a fully autonomous AI customer service agent that flips the script. We wanted to build an agent that is proactive—one that continuously monitors carrier networks, cross-references lifetime customer value in a knowledge graph, and automatically issues compensation or personalized apologies before the customer even opens a support ticket.
What it does
How we built it
JARVIS is a complex orchestration engine built to run entirely without human intervention.
The Backend (Orchestrator): Built with Python and Flask, the core of the app is a 60-second background agent loop. Knowledge Graph: We used Neo4j AuraDB as our source of truth. Instead of flat tables, we map
(Customer) -> (Order) -> (Issue) -> (Resolution) . This allows the AI to instantly query multi-hop relationships (e.g., "Has this VIP customer had a shipping issue in the last 6 months?"). The Brain: We utilized Qwen3-32B via Fastino for rapid, high-reasoning LLM decision-making. We built a dynamic prompting engine that splits into PROACTIVE and REACTIVE modes. The Tools & Integrations: Tavily Web Search API: Pulls live carrier outage and weather data into the prompt context to justify delays. Yutori Scouting & Browsing APIs: Pings carrier tracking links, and autonomously uses headless browsers to interact with UI elements (like filing a FedEx lost package claim). Shopify REST API: Directly executes financial actions for store credits and refunds. Senso: Acts as the policy engine to determine dynamic compensation rules. The Frontend: A responsive React + Vite business dashboard utilizing Socket.IO for real-time observability, alongside react-force-graph to visualize the Neo4j knowledge map live.
Challenges we ran into
AI Guardrails & Exploitation: One of our biggest concerns was an AI agent hallucinating or being exploited by customers to give away thousands of dollars in free refunds. We solved this by hard-wiring Neo4j Graph Guardrails into the prompt. The system traverses the graph to calculate the sum of previous resolutions: $$ \text{Total Credits} = \sum_{i=1}^{n} \text{Credit}_i $$ If $ \text{Total Credits} \ge $100 $, the orchestrator completely strips the LLM's financial clearance and forces an escalate state for human review. Context-Aware Routing: When customers chat, they rarely say "Where is order-1043?". They just say "Where is my shoe?". We had to build intelligent regex parsing and graph-traversal fallbacks so the LLM could cross-reference their active OrderHistory array from the database and ask the customer clarifying questions before proceeding. Real-time Observability: Making a silent background agent visible to business owners was technically difficult. We built a complex event system over WebSockets to broadcast every single step of the LLM—from policy lookup to Shopify API execution—into a live Activity Feed.
Log in or sign up for Devpost to join the conversation.