Autonomous-E-Commerce-Supply-Chain-Customer-Intelligence-Eng

E commerce supply chain

Comment

Project Breakdown: Autonomous E-Commerce Logistics & Customer Intelligence Engine

What Is This Project?

This project is a full-stack, production-oriented Machine Learning and Generative AI system designed to solve a critical e-commerce challenge: predicting package delivery delays before they happen, identifying the exact root cause, and autonomously taking operational and customer-facing corrective action.

Most machine learning pipelines in logistics stop at outputting a probability (e.g., "This shipment has an 82% risk of delay"). That leaves operations teams guessing why the delay is occurring and leaves customer service scrambling after the customer is already angry.

This engine bridges that gap by connecting three distinct technical layers:

  1. Predictive Machine Learning: Predicts service level agreement (SLA) breaches days in advance.
  2. Explainable AI (TreeSHAP): Mathematically attributes which factors are driving that specific delay (e.g., seller fulfillment backlog, route congestion, or parcel bulkiness).
  3. Autonomous GenAI Agent & RAG: Takes the mathematical explanation, retrieves relevant customer sentiment from a vector database, and automatically drafts an internal logistics intervention memo alongside an empathetic, bilingual customer apology email with a tailored discount voucher.

The Real-World Problem

In large-scale marketplaces like Brazil's Olist, thousands of independent merchants ship items across vast geographic distances using regional postal services and private couriers. This operational setup introduces major friction points:

  • Sellers take too long to pack goods: Some merchants take 4–6 days just to dispatch items to the post office.
  • Geographic complexity: Shipments traveling from industrial hubs in São Paulo to rural areas in the Northeast traverse thousands of kilometers, often through multiple sorting hubs.
  • Payment processing lags: Offline payment slips (Boleto Bancário) take 1 to 2 business days to settle before an order can even be packed.
  • Customer churn: When orders arrive late without warning, customer satisfaction drops off a cliff, leading to 1-star reviews, refund requests, and platform abandonment.

How the System Works (End-to-End Workflow)

[Raw Relational Data]
   │  (9 CSVs: orders, items, payments, products, sellers, customers, geo, reviews, translations)
   ▼
[Data Engineering & Spatial Pipeline]
   │  • Removes GPS outliers and maps 19,000+ zip codes to median coordinates.
   │  • Resolves 1-to-many relationship fan-outs (orders with multiple items/payments).
   │  • Drops physically impossible records (e.g., delivered before dispatch).
   ▼
[Feature Engineering Store]
   │  • Calculates true seller-to-customer route distances using Haversine geodesics.
   │  • Tracks seller reliability using Laplace-smoothed historical on-time rates.
   │  • Encodes cyclic temporal patterns (hour, day of week) using trigonometric sine/cosine.
   │  • Measures parcel volumetrics (density, dimensional weight vs. physical weight).
   ▼
[Predictive Modeling Core]
   │  • Tuned ensemble of LightGBM and CatBoost.
   │  • Optimized for Precision-Recall AUC (PR-AUC) to catch rare delay events (8.1% of orders).
   ▼
[Explainable AI Layer (TreeSHAP)]
   │  • Breaks down the model's decision into individual feature contributions for that order.
   ▼
[GenAI Agent & Vector Search (ChromaDB + Gemini)]
   │  • ChromaDB retrieves historical customer reviews detailing similar past failures.
   │  • Gemini 2.5 Flash reviews the SHAP drivers, historical context, and order value.
   ▼
[Automated Resolution Output]
      1. Operations Memo: Instructions to warehouse/carrier to expedite transit.
      2. Customer Retention Notice: Bilingual message + dynamically calculated credit voucher.

Detailed Component Walkthrough

1. Data Cleaning & Relational Engineering

Real-world e-commerce data is rarely clean or neatly organized into a single file. Working with Olist's 9 relational CSVs required solving several challenges:

  • Relational Join Fan-Outs: Merging orders directly with line items or payments creates duplicate rows when an order has multiple items. The pipeline aggregates items and payments at the order_id level first (calculating total price, total freight, item counts, and payment installment profiles) before merging.
  • Geospatial Coordinate Outliers: Raw geolocation data contained coordinates outside Brazil (in the ocean or other continents). The pipeline filters coordinates to Brazil's true geographic boundaries and calculates the median latitude/longitude centroid for all 19,000+ postal prefixes.
  • Chronological Cleaning: Orders logged as delivered before carrier dispatch or before payment approval are filtered out to maintain chronological integrity.

2. Key Insights from Exploratory Data Analysis (EDA)

Exploratory analysis uncovered operational patterns that shaped both the model and the recovery agent:

  • The 3-Day Satisfaction Cliff: When orders are 1 to 3 days late, average customer reviews drop moderately from 4.2 stars to 3.76 stars. However, once an order is 4 or more days late, the average score drops to 2.31 stars, and 1-star reviews jump by 380%. This proved that automated interventions must occur immediately when a delay is detected, before Day 3 is reached.
  • Regional Logistics Disparity: Orders shipped to the Northeast region experience a 14.33% delay rate with an average delivery time of 20 days, compared to 7.45% and 10.7 days for the Southeast region.
  • Boleto Clearing Lag: Orders paid via Boleto take an average of 33 hours for payment approval (versus 16 minutes for credit cards), consuming valuable fulfillment buffer time early in the process.

3. Feature Engineering Store

To prevent lookahead data leakage (using future information to predict past events), all behavioral metrics use expanding historical windows:

  • Laplace-Smoothed Seller Reliability: Instead of a simple average on-time rate—which gives misleading 0% or 100% scores to new sellers with only 1 order—the system uses Laplace smoothing with a global prior ($m=5$) to pull low-volume sellers toward the platform average.
  • Cyclic Trigonometric Encodings: Hours of the day ($0\text{ to }23$) and days of the week ($0\text{ to }6$) are converted into continuous $(\sin, \cos)$ pairs so the machine learning trees recognize that 23:00 and 00:00 are consecutive hours.
  • Logistics & Packaging Features: Calculates volumetric parcel density ($\text{g/cm}^3$), freight cost per kilometer, and interstate corridor risk flags.

4. Predictive Modeling & Cost-Sensitive Optimization

Delivery delay occurs in roughly 8.11% of delivered orders. Evaluating standard accuracy on an imbalanced dataset is misleading because a model that always predicts "on-time" would still be 91.9% accurate while catching 0 delays.

  • Algorithms: A blended ensemble of LightGBM (histogram-based tree splitting) and CatBoost (symmetric decision trees).
  • Optimization: Tuned using Optuna Bayesian Optimization over 50 trials across learning rate, tree depth, regularization, and scale-positive class weights.
  • Evaluation: Optimized for PR-AUC (Precision-Recall Area Under Curve) and calibrated to maximize the F2-Score, prioritizing catching as many true delays as possible (high recall) while controlling false alarms.

5. Explainable AI with TreeSHAP

Instead of treating the gradient boosting model as a black box, the system computes exact TreeSHAP (SHapley Additive exPlanations) values for every prediction. This calculates the precise numerical push (in log-odds) that each feature contributed toward the delay prediction.

For example, for a flagged order, TreeSHAP can isolate:

  • seller_avg_dispatch_lag_days = 4.8 contributed $+0.38$ to the delay risk.
  • lane_historical_delay_rate = 0.24 contributed $+0.45$ to the delay risk.
  • volumetric_weight_kg = 6.85 contributed $+0.18$ to the delay risk.

6. Generative AI & Semantic RAG Layer

The top positive SHAP drivers are packaged into a structured JSON object and handed to the GenAI layer:

  • Vector Database (ChromaDB): 40,000+ Portuguese customer reviews are embedded using a multilingual sentence transformer (paraphrase-multilingual-MiniLM-L12-v2).
  • Semantic Context Retrieval (RAG): When an order in furniture_decor is at risk due to seller dispatch delays, the vector database retrieves similar past complaints from that category and region.
  • Gemini 2.5 Flash Agent: Acting as an autonomous Chief Logistics Officer, Gemini reads the SHAP drivers, the order price, and the customer complaints, then generates:
  • An Executive Root Cause Diagnostic explaining the operational failure.
  • A Logistics Corrective Action (e.g., seller priority dispatch alert, direct line-haul re-routing).
  • An Empathetic Bilingual Customer Notification (Portuguese and English) with a dynamically calculated compensation voucher based on order value and delay severity.

7. Interactive Gradio Application

The entire pipeline is wrapped in a multi-tab Gradio web interface:

  • Live SLA Simulator: Sliders allow users to tweak product dimensions, prices, routes, and seller history. The interface immediately calculates breach probability, renders an interactive TreeSHAP waterfall plot, and displays the autonomous remediation plan generated by Gemini.
  • Customer Feedback Explorer: A natural language search tab that queries the ChromaDB vector database to inspect customer sentiment across categories and Brazilian states.

What Makes This Project Stand Out

  • Solves the Complete Problem: Goes beyond predictive modeling to address automated diagnosis, operational decision-making, and customer retention.
  • Zero Data Leakage: Built with chronological expanding-window splits and Bayesian priors, matching real-world data engineering standards.
  • Interpretable Generative AI: Avoids unstructured LLM hallucinations by grounding the agent directly in mathematical SHAP attribution vectors and verified historical reviews.

Built With

  • jupyter-notebook
Share this project:

Updates