AI-Delivery-Route-Optimizer

AI-powered Delivery Route Optimizer using Reinforcement Learning (Q-Learning) with a custom Warehouse Environment and Streamlit dashboard.

Comment
# 📦 AI Delivery Route Optimizer ![AI Delivery Route Optimizer Banner](assets/banner.png) [![Python Version](https://img.shields.io/badge/Python-3.10%2B-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/) [![Streamlit](https://img.shields.io/badge/Streamlit-1.30%2B-FF4B4B?style=for-the-badge&logo=streamlit&logoColor=white)](https://streamlit.io/) [![Gymnasium](https://img.shields.io/badge/Gymnasium-0.29%2B-000000?style=for-the-badge)](https://gymnasium.farama.org/) [![NumPy](https://img.shields.io/badge/NumPy-1.24%2B-013243?style=for-the-badge&logo=numpy&logoColor=white)](https://numpy.org/) [![License](https://img.shields.io/badge/License-MIT-4CAF50?style=for-the-badge)](LICENSE) An advanced Reinforcement Learning (RL) pathfinder and route optimization platform. The system trains off-policy **Q-Learning** agents to discover optimal delivery routes inside standardized environments (`Taxi-v4`) and custom warehouse grids containing obstacles. 🚀 **[Streamlit Live Demo](https://ai-delivery-route-optimizer-bx9jagmykhfdqxhlrnmsvu.streamlit.app/)**  |  📂 **[GitHub Repository](https://github.com/Malik7343/AI-Delivery-Route-Optimizer)**

📖 Overview

The AI Delivery Route Optimizer showcases how tabular Q-learning agents can be leveraged to resolve complex pathfinding challenges under constraint-heavy conditions.

The optimizer supports two primary routing spaces:

  1. Taxi Navigation (Taxi-v4): The classic Gym environment where an agent must navigate a grid to pick up a passenger and deliver them to a destination with minimal steps.
  2. Warehouse Grid World (WarehouseDeliveryEnv): A custom grid environment simulating shelves (obstacles), loading docks (depots), and random delivery locations. A BFS path validator ensures reachable targets at initialization.

Both environments compile routes and stream the live simulation directly to a Vercel-style, minimalist dark theme dashboard.

✨ Features

  • 🖥️ Minimalist Dark Dashboard: A premium, responsive interface featuring layout controls and smooth state animations.
  • 🚚 Custom Warehouse Environment: Configurable grid parameters with dynamic goal selections and static shelf layouts.
  • 🚖 Taxi-v4 Agent Support: Out-of-the-box support for gymnasium standards.
  • 📈 Live Metrics Streaming: Frame-by-frame stats for steps taken, reward progression, status, and processing times.
  • 📊 Smoothed Reward Graphing: Renders Matplotlib training progression metrics using rolling window averages directly on the main dashboard screen.
  • 🛡️ BFS Path Validation: Pre-validates custom layouts to guarantee that obstacles never block the delivery path.

📸 Screenshots

Project Dashboard View

Dashboard View

Warehouse Route Simulator (Active Simulation)

Warehouse Simulation

Matplotlib Reward Curves

Reward Curve

🎥 Demonstration Video

(To generate a demo.gif recording, run the app locally using streamlit run app.py and record the browser window traversing paths using a tool like ScreenToGif, then overwrite the placeholder below).

Demo Simulation Route

🏗️ Architecture

The project maintains a decoupling between environment layout, decision-making agents, and visual representations:

graph TD
    A[Streamlit Web UI] -->|Trigger Run| B[Simulation Loop]
    B -->|State Lookup| C[Q-Table Weights]
    B -->|Action step| D[Gymnasium / Custom Env]
    D -->|Feedback State/Reward| B
    B -->|HTML Render String| A
  • Agents: Standard Q-learning agent featuring $\epsilon$-greedy action selections, learning rate updates, and decay mechanisms.
  • Environments: Extends standard gymnasium.Env interfaces with state space flattening.
  • Dashboard Helpers: Inject CSS sheets and compile ASCII-to-HTML grid views.

📂 Project Structure

delivery-route-optimizer/
├── agents/
│   └── q_learning_agent.py      # Q-learning Decision Engine
├── environments/
│   └── warehouse_env.py         # Custom Grid World Gym Env
├── saved_models/
│   ├── taxi_q_table.npy         # Trained Taxi weights
│   ├── training_history.npy     # Taxi reward curve metrics
│   ├── warehouse_q_table.npy    # Trained Warehouse weights
│   ├── warehouse_obstacles.npy  # Saved obstacle layout
│   └── warehouse_training_history.npy # Warehouse reward curve metrics
├── training/
│   ├── train_taxi.py            # Training script for Taxi env
│   ├── train_warehouse.py       # Training script for Warehouse env
│   ├── evaluate_agent.py        # evaluation loops
│   └── visualize_training.py    # Offline reward charting
├── utils/
│   └── dashboard_helpers.py     # HTML render templates & CSS injection
├── assets/
│   ├── banner.png               # Repository banner
│   ├── dashboard.png            # Main layout screenshot
│   ├── taxi_simulation.png      # Taxi navigation view
│   ├── warehouse_simulation.png # Warehouse navigation view
│   └── reward_history.png       # Matplotlib reward plot
├── app.py                       # Streamlit dashboard portal
├── requirements.txt             # Project library requirements
├── LICENSE                      # Open source license (MIT)
├── test_setup.py                # environment test script
└── README.md                    # Project documentation

⚙️ Installation Guide

Follow these steps to configure your local development workspace:

1. Prerequisite

Ensure you have Python 3.10 or newer installed.

2. Clone and Setup Environment

# Clone the repository
git clone https://github.com/Malik7343/AI-Delivery-Route-Optimizer.git
cd delivery-route-optimizer

# Initialize Python Virtual Environment
python -m venv .venv

3. Activate Virtual Environment

  • Windows (PowerShell): powershell .venv\Scripts\Activate.ps1
  • Unix/macOS: bash source .venv/bin/activate

4. Install Dependencies

pip install --upgrade pip
pip install -r requirements.txt

5. Verify Installation

Ensure the environment loads successfully by executing:

python test_setup.py

🚀 Usage Guide

1. Training the Agents

To train the agents from scratch and save the model weights (.npy files), execute the training scripts:

# Train Taxi agent (10,000 episodes)
python training/train_taxi.py

# Train Warehouse agent (20,000 episodes)
python training/train_warehouse.py

2. Launching the Web Portal

Start the local Streamlit service to run simulations interactively:

streamlit run app.py

Open http://localhost:8501 in your browser to view the SaaS dashboard.

🎮 Demo & Interface Controls

  • 🚖 Taxi Simulation Button: Instantly loads saved_models/taxi_q_table.npy, sets up a new Taxi-v4 environment instance, and runs a greedy route evaluation.
  • 📦 Warehouse Simulation Button: Instantly loads the trained weights and fixed shelf obstacle coordinates to run the pathfinder grid simulation.
  • Metrics Panel: Updates dynamically inside the card grid showing the cumulative reward progression step-by-step.

📊 Results and Metrics

Environment Training Episodes Success Rate (Greedy) Optimal Average Reward
Taxi-v4 10,000 100% +7.32
Warehouse-v1 (8x8) 20,000 100% +42.39

🛠️ Technology Stack

🔮 Future Improvements

  • [ ] Deep Q-Networks (DQN): Implement neural network function approximators to solve continuous/large state spaces.
  • [ ] Dynamic Obstacles: Introduce moving warehouse bots as dynamic hazards.
  • [ ] Multi-Agent Systems: Support coordination between multiple delivery trucks.

📄 License

Distributed under the MIT License. See LICENSE for more details.

👥 Authors

  • Malik - Lead AI Engineer & UX Architect - GitHub

Built With

Share this project:

Updates