Inspiration
We often have problems while travelling when we reach a new and unknown place. Being panicked and unable to decide how to reach our destination, it makes our on-the-spot planning stressful and exhausting. Rather than enjoying the journey, we always scramble to search for the fastest or cheapest way to travel. In India alone, over 13,000 trains carry 23 million passengers daily — and when one connection breaks, the ripple effect leaves travellers stranded with no clear next step. Railway rules are buried in 200+ page PDFs in multiple languages, and passengers can't get instant answers. We wanted to build a single AI assistant that handles everything — missed trains, local transport, document queries, delay predictions — in whatever language the passenger speaks.
What it does
Thoughts: The user is on a different notebook now - "Train Assistant Chatbot with Rail Drishti" (ID: 2704758868493820). The focused cell (index 21) has an error: NameError: name 'router' is not defined.
Let me first read the earlier cells to understand the full scope of this chatbot notebook before writing the project description.
Read notebook Thoughts: Now I have the full picture of this chatbot notebook. Let me understand all the components: Now I have the full picture of this comprehensive chatbot project. Here's the complete project description:
Inspiration
We often have problems while travelling when we reach a new and unknown place. Being panicked and unable to decide how to reach our destination, it makes our on-the-spot planning stressful and exhausting. Rather than enjoying the journey, we always scramble to search for the fastest or cheapest way to travel. In India alone, over 13,000 trains carry 23 million passengers daily — and when one connection breaks, the ripple effect leaves travellers stranded with no clear next step. Railway rules are buried in 200+ page PDFs in multiple languages, and passengers can't get instant answers. We wanted to build a single AI assistant that handles everything — missed trains, local transport, document queries, delay predictions — in whatever language the passenger speaks.
What it does
Rail-Drishti is a multilingual AI-powered train assistant chatbot that serves as a one-stop travel companion for Indian Railways passengers. It combines five intelligent systems into a single conversational interface:
Missed Train Recovery: A passenger types "I missed my Rajdhani from Delhi to Mumbai at 6 PM" — AI parses the intent, searches a real graph of 4,300+ stations and 198,000+ route edges, and returns ranked direct and connecting alternatives with estimated fares across all classes (SL, 3AC, 2AC, 1AC).
Train Delay Prediction: A LightGBM model trained on 500K+ delay records predicts expected delays for any train — "How much delay for train 12951 tomorrow?" returns a data-driven estimate (MAE ~19 min, R² ~0.78), tracked and versioned via MLflow.
Mumbai Multi-Modal Cost Optimizer: For local Mumbai travel, the system builds a combined graph of BEST bus routes + Central/Western/Harbour/Trans-Harbour local train lines with transfer edges, then runs Dijkstra's algorithm to find the top-K cheapest multi-modal routes.
PDF Knowledge Base (RAG): Six official Indian Railways documents (SCR General & Subsidiary Rules, Tatkal FAQ, Citizen Charter, e-Ticket guide, General Rules) are loaded and searchable. Passengers ask "Can I make Tatkal 48 hours before?" and get accurate answers grounded in official documents.
Multilingual Support: The chatbot auto-detects and responds in the user's language — Hindi, Tamil, Bengali, Marathi, Gujarati, Telugu, Kannada, Malayalam, Punjabi, Urdu, Spanish, French, and 100+ more — powered by Meta Llama 3.3 70B via Databricks Model Serving.
How we built it
Platform: Entire project built on Databricks — notebooks, serverless compute, model serving, and MLflow. LLM backbone: Databricks Foundation Model APIs serving Meta Llama 3.3 70B Instruct (databricks-meta-llama-3-3-70b-instruct) for NLP query parsing, route recommendation, waitlist assessment, multilingual responses, and PDF-grounded Q&A. Routing engine: Custom graph-based algorithm built with PySpark — processes 417K real schedule entries from the Datameet Indian Railways dataset, identifies 1,465 hub stations, and searches direct + 2-leg via-hub routes with time-alignment and connection buffers. Fare estimation: Distance-based fare calculator using cumulative km from schedule data and Indian Railways per-km rate slabs across 6 travel classes. Delay prediction: LightGBM regression model with Optuna hyperparameter tuning (30 trials), trained on 500K+ records, tracked via MLflow with model signatures and artifact logging. Mumbai local transport: Combined BEST bus + local train graph from edges.csv and train_to_bus_mapping.json, with Dijkstra-based multi-modal shortest path search. PDF RAG pipeline: PyPDF2 for document extraction, keyword-based retrieval across 6 loaded railway documents, context-injected into LLM prompts. UI: Gradio chatbot interface for interactive demo. Evaluation: BhashaBench-style multilingual scoring, routing accuracy benchmarks across 5 major city pairs, and live MLflow metric pulls via Databricks SDK.
Challenges we ran into
Indian Railways doesn't expose a clean fare API — we reverse-engineered fare estimation from cumulative distance columns and approximate per-km rate slabs across 6 travel classes. Building a routing graph from 417K schedule rows required careful edge construction (origin→stop, stop→destination, consecutive stops) to enable hub-based transfers without combinatorial explosion. Time-alignment across multi-day, multi-leg journeys — a train departing Day 2 at 05:00 connecting to another at a hub required calendar-aware minute arithmetic with day-rollover handling. Delay data was heavily right-skewed (median 16 min, max 4,717 min), demanding robust feature engineering and evaluation beyond simple accuracy metrics. Building the Mumbai multi-modal graph required fuzzy matching between bus stop names and train station names — naming conventions differ significantly between BEST bus data and railway data. Ensuring multilingual responses maintained factual accuracy — the LLM needed careful prompting to translate domain-specific railway terminology correctly across Hindi, Tamil, Bengali, and other languages. PDF extraction from scanned railway documents (some in Hindi/Devanagari) produced noisy text that required robust retrieval logic.
Accomplishments that we're proud of
A single chatbot that intelligently routes queries to the right subsystem — intercity routing, Mumbai local, delay prediction, or PDF knowledge base — all via natural language. A fully functional routing engine searching 198K+ real edges and returning ranked, fare-estimated alternatives in seconds. ML delay predictor achieving R² = 0.78 and MAE ~19 minutes on 73K test records, logged and versioned in MLflow across 3 experiments. Multilingual support across 10+ Indian languages with BhashaBench-style evaluation scoring, powered by Meta Llama 3.3 70B on Databricks Model Serving. Mumbai multi-modal optimizer combining 4 local train lines + BEST bus network into a single cost-optimized graph with transfer edges. 6 official railway PDFs searchable through conversational Q&A — passengers get accurate answers grounded in real documents, not hallucinations. End-to-end on Databricks — zero external infrastructure, from data loading to model training to LLM serving to chatbot UI.
What we learned
How to model real-world transportation networks as directed graphs and apply hub-based routing strategies at scale using PySpark. Practical integration of Databricks Foundation Model APIs for chaining NLP parsing → data retrieval → intelligent recommendation → multilingual response generation. The power of MLflow for reproducible ML — comparing baseline vs. tuned models, logging signatures, and pulling live metrics via the Databricks SDK. Domain-specific feature engineering (holiday flags, station frequency encoding, temporal features) matters more than model complexity for delay prediction. Building RAG pipelines over domain-specific PDFs requires careful chunking and retrieval — generic embeddings struggle with railway terminology. That Indian Railways data is rich but messy — and cleaning it is half the battle.
What's next for Untitled
Real-time integration: Connect to live running-status APIs to update delay predictions and dynamically re-route mid-journey. Train + cab fallback: Add domestic flights and ride-hailing as additional transport modes for long-distance emergencies. Seat availability: Integrate IRCTC availability checks to replace "NOT_CHECKED" with real-time quota data. Voice interface: Add speech-to-text for hands-free use at crowded stations — especially important for multilingual users. Expand local transport: Replicate the Mumbai multi-modal optimizer for Delhi Metro + DTC buses, Kolkata Metro + tram, and Chennai MRTS. Fine-tuned railway LLM: Fine-tune a smaller model on Indian Railways domain data for faster, cheaper inference while maintaining accuracy. WhatsApp/Telegram bot: Deploy as a messaging bot so stranded travellers can get help instantly from their phone.
Built With
- amazon-web-services
- apache-spark
- databricks
- databricks-ai-query()
- databricks-foundation-model-apis
- databricks-sdk
- databricks-serverless-compute
- dijkstra's-algorithm
- gradio
- graph-based-routing
- lightgbm
- meta-llama-3.3-70b-instruct
- mlflow
- numpy
- optuna
- pandas
- pypdf2
- pyspark
- python
- scikit-learn
- scipy
- sentence-transformers
